Commit Graph

918 Commits (c455a314fd2faa5299b93b47b44d9eb6b3293321)

Author SHA1 Message Date
Eugen Rochko 5f591d979c
Fix non-numbers being passed to Redis and causing an error (#11697) 6 years ago
Eugen Rochko b54b725d6b
Fix uncaught domain normalization error in remote follow (#11703) 6 years ago
Eugen Rochko cb447b28c4
Add profile directory to web UI (#11688)
* Add profile directory to web UI

* Add a line of bio to the directory
6 years ago
Eugen Rochko 73ca0bb925
Add option to include reported statuses in warning e-mail (#11639) 6 years ago
Eugen Rochko 282ea17078
Add soft delete for statuses for instant deletes through API (#11623)
* Add soft delete for statuses to allow them to appear instant

* Allow reporting soft-deleted statuses and show them in the admin UI

* Change index for getting an account's statuses
6 years ago
Eugen Rochko 97192d9a77
Fix remote and staff-removed statuses leaving media behind for a day (#11638)
The reason for unattaching media instead of removing it is to support
delete & redraft functionality, but remote or staff-removed statuses
will never be redrafted, so the media should be deleted immediately
6 years ago
Eugen Rochko dff46b260b
Fix ignoring whole status because of one invalid hashtag (#11621)
Fix #11618
6 years ago
ThibG cb62a83a71 Add invite comments (#10465) 6 years ago
ThibG 9b6a5ed109 Add public blocks to /about/blocks (#11298)
* Add automatic blocklist display in /about/blocks

Inspired by https://github.com/Gargron/mastodon.social-misc

* Add admin option to set who can see instance blocks

* Normalize locales files

* Rename “Sandbox” to “Silence” for consistency

* Disable /about/blocks when in whitelist mode

* Optionally display rationale for domain blocks

* Only display domain blocks that have user-facing limitations, and order them

* Redesign table of blocked domains to better handle long domain names and rationales

* Change domain blocks ordering now that rationales aren't displayed right away

* Only show explanation for block severities actually in use

* Reword instance block explanations and add disclaimer for public fetch mode
6 years ago
Eugen Rochko 96702e7f67
Add `tootctl cache recount` command (#11597) 6 years ago
Eugen Rochko cc0a55cf9a
Add more accurate hashtag search (#11579)
* Add more accurate hashtag search

Using ElasticSearch to index hashtags with edge n-grams and score
them by usage within the last 7 days since last activity. Only
hashtags that have been reviewed and are listable can appear in
searches, unless they match the query exactly

* Fix search analyzer dropping non-ascii characters
6 years ago
Stanislas 5d8ee24cd5 Remove WebP support (#11589) 6 years ago
Eugen Rochko 8fdff2748f
Add more accurate account search (#11537)
* Add more accurate account search

When ElasticSearch is available, a more accurate search is implemented:

- Using edge n-gram index for acct and display name
- Using asciifolding and cjk width normalization on display names
- Using Gaussian decay on account activity for additional scoring (recency)
- Using followers/friends ratio for additional scoring (spamminess)
- Using followers number for additional scoring (size)

The exact match precedence only takes effect when the input conforms
to the username format and the username part of it is complete, i.e.
when the user started typing the domain part.

* Support single-letter usernames

* Fix tests

* Fix not picking up account updates

* Add weights and normalization for scores, skip zero terms queries

* Use local counts for accounts index, adjust search parameters

* Fix mistakes

* Using updated_at of accounts is inadequate for remote accounts
6 years ago
Arnout Engelen d7bdddbeef Include max image dimensions in error (#11552) 6 years ago
kedama aebefc7ce9 Fix some emojis in profile metadata labels are not emojified. (#11534) 6 years ago
Eugen Rochko 7a1f8a58df
Fix crash when saving invalid domain name (#11528)
Fix #7629
6 years ago
Eugen Rochko 898dacf83e
Fix weekly usage not being displayed correctly in hashtag admin UI (#11524)
Fix percentages in usage breakdown having too many digits

Change trending hashtags to only ask for review if a hashtag enters
the top 3 position in the set, since it's the only items shown in
the default web UI
6 years ago
Eugen Rochko 41aac1ba3a
Add GIF and WebP support for custom emojis (#11519)
Fix #11466
6 years ago
Eugen Rochko f51c7c105f
Fix acct URIs with IDN domains not being resolved (#11520)
Fix #11494
6 years ago
ThibG bced70469a Add domain block notes (#11515)
* Add database columns for adding notes to domain blocks/restrctions

* Add admin UI to set private and public comments when blocking a domain

* Add text for private and public comments on domain blocks

* Show domain block comments in admin UI

* Add comments to the domain block undo page

* Make UnblockDomainService more robust regarding upgraded domain blocks

* Allow editing domain blocks

* Rename button from “undo domain block” to “view domain block” in account admin UI

* Change test to unsilence silenced users from upgraded blocks
6 years ago
Eugen Rochko 94c54997cf
Fix trending tags returning less items than requested sometimes (#11513)
Add better sorting defaults to the hashtags admin UI

Add "not reviewed" filter to hashtags admin UI
6 years ago
Eugen Rochko ac33f1aedd
Fix account tags not being saved correctly (#11507)
* Fix account tags not being saved correctly

Regression from f371b32

Fix Tag#discoverable not returning tags where listable is nil instead of true

Add notice when saving hashtags in admin UI

Change public hashtag and directory pages to return 404 for forbidden tags

* Remove unused locale string
6 years ago
Eugen Rochko a8958d07e2
Fix featured tags not finding the right tag on save (#11504)
Regression from f371b32
6 years ago
Eugen Rochko 9072fe5ab6
Add trends UI with admin and user settings (#11502) 6 years ago
Eugen Rochko 115dab78f1
Change admin UI for hashtags and add back whitelisted trends (#11490)
Fix #271

Add back the `GET /api/v1/trends` API with the caveat that it does
not return tags that have not been allowed to trend by the staff.

When a hashtag begins to trend (internally) and that hashtag has
not been previously reviewed by the staff, the staff is notified.

The new admin UI for hashtags allows filtering hashtags by where
they are used (e.g. in the profile directory), whether they have
been reviewed or are pending reviewal, they show by how many people
the hashtag is used in the directory, how many people used it
today, how many statuses with it have been created today, and it
allows fixing the name of the hashtag to make it more readable.

The disallowed hashtags feature has been reworked. It is now
controlled from the admin UI for hashtags instead of from
the file `config/settings.yml`
6 years ago
Eugen Rochko 29609c09f4
Fix tag score not being updated when the tag is trending (#11465) 6 years ago
Eugen Rochko 92de439c04
Change hashtag search to only return results that have trended in the past (#11448)
* Change hashtag search to only return results that have trended in the past

A way to eliminate typos and other one-off "junk" results

* Fix excluding exact matches that don't have a score

* Fix tests
6 years ago
Eugen Rochko 648cdbc04a
Add hashtag score for better sorting of autosuggestions (#11427)
* Add hashtag score for better sorting of autosuggestions

* Do not use `~<~` operator with no text_pattern_ops index
6 years ago
Eugen Rochko 24552b5160
Add whitelist mode (#11291) 6 years ago
Eugen Rochko e136112ab7
Fix tag normalization and migration not removing duplicate tags (#11441)
Fix #11428
6 years ago
Eugen Rochko b9b0313c78
Revert "Remove conversation URI (#11423)" (#11424)
This reverts commit 75f7f9930e.
6 years ago
Eugen Rochko 75f7f9930e
Remove conversation URI (#11423)
It is not part of ActivityPub and will free up a lot of space
6 years ago
Eugen Rochko f371b32137
Change hashtags to preserve first-used casing (#11416) 6 years ago
Eugen Rochko 501148ab91
Remove timestamps from converted images to make them deterministic (#11408) 6 years ago
ThibG 92569ffde8 Fix invites not being disabled upon account suspension (#11412)
* Disable invite links from disabled/suspended users

* Add has_many invites relationship to users

* Destroy unused invites when suspending an account
6 years ago
Eugen Rochko 964ae8eee5
Change unconfirmed user login behaviour (#11375)
Allow access to account settings, 2FA, authorized applications, and
account deletions to unconfirmed and pending users, as well as
users who had their accounts disabled. Suspended users cannot update
their e-mail or password or delete their account.

Display account status on account settings page, for example, when
an account is frozen, limited, unconfirmed or pending review.

After sign up, login users straight away and show a simple page that
tells them the status of their account with links to account settings
and logout, to reduce onboarding friction and allow users to correct
wrongly typed e-mail addresses.

Move the final sign-up step of SSO integrations to be the same
as above to reduce code duplication.
6 years ago
Eugen Rochko bd87e66679
Remove WebSub subscriptions (#11303) 6 years ago
ThibG c37c1da41e Disallow numeric-only hashtags (#11363)
* Add spec covering numeric-only hashtags

* Fix hashtag regex
6 years ago
ThibG 730c4053d6 Add ActivityPub actor representing the entire server (#11321)
* Add support for an instance actor

* Skip username validation for local Application accounts

* Add migration script to create instance actor

* Make Codeclimate happy

* Switch to id -99 for instance actor

* Remove unused `icon` and `image` attributes from instance actor

* Use if/elsif/else instead of return + ternary operator

* Add instance actor to fresh installs

* Use instance actor as instance representative

Use instance actor for forwarding reports, relay operations, and spam
auto-reporting.

* Seed database in test environment

* Fix single-user mode

* Fix tests

* Fix specs to accomodate for an extra `Account`

* Auto-reject follows on instance actor

Following an instance actor might make sense, but we are not handling that
right now, so auto-reject.

* Fix webfinger lookup and serialization for instance actor

* Rename instance actor

* Make it clear in the HTML view that the instance actor should not be blocked

* Raise cache time for instance actor as there's no dynamic content

* Re-use /about/more with a flash message for instance actor profile
6 years ago
Eugen Rochko 84e988479e
Fix only one middle dot being recognized in hashtags (#11345)
Fix #10934
6 years ago
Eugen Rochko 4906cabc6b
Add aac, m4a, 3gp to allowed audio formats (#11342)
Fix #11186
6 years ago
ThibG 7e2b6da57f Add setting to disable the anti-spam (#11296)
* Add environment variable to disable the anti-spam

* Move antispam setting to admin settings

* Fix typo

* antispam → spam_check
6 years ago
ThibG 91544a6cb5 Remove unused Account#magic_key (#11327) 6 years ago
Eugen Rochko 9b1d3e4acb
Add option to disable real-time updates in web UI (#9984)
Fix #9031
Fix #7913
6 years ago
Eugen Rochko 6ff67be0f6
Add a spam check (#11217)
* Add a spam check

* Use Nilsimsa to generate locality-sensitive hashes and compare using Levenshtein distance

* Add more tests

* Add exemption when the message is a reply to something that mentions the sender

* Use Nilsimsa Compare Value instead of Levenshtein distance

* Use MD5 for messages shorter than 10 characters

* Add message to automated report, do not add non-public statuses to
automated report, add trust level to accounts and make unsilencing
raise the trust level to prevent repeated spam checks on that account

* Expire spam check data after 3 months

* Add support for local statuses, reduce expiration to 1 week, always create a report

* Add content warnings to the spam check and exempt empty statuses

* Change Nilsimsa threshold to 95 and make sure removed statuses are removed from the spam check

* Add all matched statuses into automatic report
6 years ago
ThibG 1e7187f2a8 Fix Status.remote scope matching *all* statuses (#11265) 6 years ago
Eugen Rochko 63c7fe8e48
Refactor controllers for statuses, accounts, and more (#11249) 6 years ago
Eugen Rochko b851456139
Remove Atom feeds and old URLs in the form of `GET /:username/updates/:id` (#11247) 6 years ago
Eugen Rochko 23aeef52cc
Remove Salmon and PubSubHubbub (#11205)
* Remove Salmon and PubSubHubbub endpoints

* Add error when trying to follow OStatus accounts

* Fix new accounts not being created in ResolveAccountService
6 years ago
Eugen Rochko ae003d54f9
Fix option to send e-mail notification about account action always being true (#11242) 6 years ago
Eugen Rochko 3fd6ab99e6
Remove deprecated REST API `GET /api/v1/timelines/direct` (#11212) 6 years ago
ThibG 082b3b282f Fix expiration date of filters being set to “Never” when editing them (#11204)
When editing a custom filter, select the shortest preset duration that
still covers the remaining time of that filter.

Fixes #9506
6 years ago
Eugen Rochko 06424786a1
Fix support for MP4 files that are actually M4V files (#11210)
Resolve #11187
6 years ago
Eugen Rochko e64e6a03dd
Add categories for custom emojis (#11196)
Fix #7940
6 years ago
ThibG 3086c645fd Add option to disable blurhash previews (#11188)
* Add option to disable blurhash previews

* Update option text

* Change options order
6 years ago
Eugen Rochko b927bb3f07
Fix audio-only OGG and WebM files not being processed as such (#11151)
Also, because Chrome sends audio/mp3 instead of audio/mpeg as it's
supposed to, we need to whitelist that mime type as well
6 years ago
Eugen Rochko aa9b37822b
Fix audio not being downloaded from remote servers (#11145) 6 years ago
Eugen Rochko 1b052c7b2d
Remove expensive counters from federation page in admin UI (#11139) 6 years ago
Eugen Rochko 707ddf7808
Change domain blocks to automatically support subdomains (#11138)
* Change domain blocks to automatically support subdomains

If a more authoritative domain is blocked (example.com), then the
same block will be applied to a subdomain (foo.example.com)

* Match subdomains of existing accounts when blocking/unblocking domains

* Improve code style
6 years ago
Eugen Rochko 49ebda4d49
Change audio format from ogg to mp3 for wider compatibility (#11141)
* Change audio format from ogg to mp3 for wider compatibility

* Add media description as title to links of unknown media attachments
6 years ago
Eugen Rochko 8f23726918
Fix converted media being saved with original extension and mime type (#11130) 6 years ago
Eugen Rochko 7696f77245
Add moderation API (#9387)
Fix #8580
Fix #7143
6 years ago
Eugen Rochko f7f23b4a19
Add audio uploads (#11123)
* Add audio uploads

Fix #4827

Accept uploads of OGG, WAV, FLAC, OPUS and MP3 files, and converts
them to OGG. Media attachments get a new `audio` type. In the UI,
audio uploads are displayed identically to video uploads.

* Improve code style
6 years ago
ThibG 17747e2cd7 Fix User#active scope only returning suspended users (#11111)
Fix a regression from #10660
6 years ago
Eugen Rochko 6a9a759f40
Change reblogs counter to be updated when boosted privately (#10964) 6 years ago
Eugen Rochko f2b743e715
Refactor all ActivityPub deliveries to be serialized and signed through one concern (#10966) 6 years ago
Eugen Rochko 1e5532e693
Add responsive panels to the single-column layout (#10820)
* Add responsive panels to the single-column layout

* Fixes

* Fix not being able to save the preference

* Fix code style issues

* Set max-height on the compose textarea and add a link to relationship manager
6 years ago
Paul Woolcock 0c933c1b8c Add `account_id` param to `GET /api/v1/notifications` (#10796)
* Add `from_account` to notifications API

this adds the ability to filter notifications by the account they
originated from

* passing a non-existent user should cause none to be returned

* Fix codeclimate warnings

* fix more codeclimate warnings

* make requested changes:

* use account id instead of user@domain
* name the param `account_id` instead of `from_account`

* Don't use `return` in a lambda
6 years ago
trwnh a6caf919e2 Change bio limit from 160 to 500 (#10790)
* Change note_length validator from 160 to 500

* Change input maxlength from 160 to 500

* update bio test from 160 to 500

* Multiply a string 30 times instead of 10
6 years ago
ThibG ae18386558 Fix “invited by” not showing up for invited accounts in admin interface (#10791) 6 years ago
ThibG 14f6ce2885 Record account suspend/silence time and keep track of domain blocks (#10660)
* Record account suspend/silence time and keep track of domain blocks

* Also unblock users who were suspended/silenced before dates were recorded

* Add tests

* Keep track of suspending date for users suspended through the CLI

* Show accurate number of accounts that would be affected by unsuspending an instance

* Change migration to set silenced_at and suspended_at

* Revert "Also unblock users who were suspended/silenced before dates were recorded"

This reverts commit a015c65d2d.

* Switch from using suspended and silenced to suspended_at and silenced_at

* Add post-deployment migration script to remove `suspended` and `silenced` columns

* Use Account#silence! and Account#suspend! instead of updating the underlying property

* Add silenced_at and suspended_at migration to post-migration

* Change account fabricator to translate suspended and silenced attributes

* Minor fixes

* Make unblocking domains always retroactive
6 years ago
Jeong Arm ce8635605c Record deleted(by mod) status to prevent re-appear (#10732)
* Record deleted(by mod) status to prevent re-appear

* Move to Tombstone

* Add missing migration script
6 years ago
Eugen Rochko 7cb369d4c6
Change e-mail whitelist/blacklist to not be checked when invited (#10683)
* Change e-mail whitelist/blacklist to not be checked when invited

And only when creating an account, not when updating it later

Fix #10648

* Fix test
6 years ago
ThibG 011b032300 Provide a link to existing domain block when trying to block an already-blocked domain (#10663)
* When trying to block an already-blocked domain, provide a link to the block

* Fix styling for links in flash messages

* Allow blocks to be upgraded but not downgraded
6 years ago
ThibG 21a73c52a7 Check that an invite link is valid before bypassing approval mode (#10657)
* Check that an invite link is valid before bypassing approval mode

Fixes #10656

* Add tests

* Only consider valid invite links in registration controller

* fixup
6 years ago
Eugen Rochko e18786dec7
Fix approved column being set to nil instead of false (#10642)
Fix https://github.com/tootsuite/mastodon/pull/10621#issuecomment-487316619
6 years ago
Eugen Rochko fba96c808d
Add blurhash (#10630)
* Add blurhash

* Use fallback color for spoiler when blurhash missing

* Federate the blurhash and accept it as long as it's at most 5x5

* Display unknown media attachments as blurhash placeholders

* Improve style of embed actions and spoiler button

* Change blurhash resolution from 3x3 to 4x4

* Improve dependency definitions

* Fix code style issues
6 years ago
Eugen Rochko e451ba0e83
Fix LDAP/PAM/SAML/CAS users not being approved instantly (#10621) 6 years ago
Eugen Rochko a9f130b8d8
Fix Keybase verification using wrong domain for remote accounts (#10547) 6 years ago
Alex Gessner d431c810d3 increase allowable length of remote proof username (#10546) 6 years ago
Eugen Rochko 400397c0c7
Change HTML validator to ignore all errors except unmatched tags (#10534) 6 years ago
Eugen Rochko 36b39fbac5
Add preference to disable e-mails about new pending accounts (#10529) 6 years ago
Eugen Rochko 8b69a66380 Add "why do you want to join" field to invite requests (#10524)
* Add "why do you want to join" field to invite requests

Fix #10512

* Remove unused translations

* Fix broken registrations when no invite request text is submitted
6 years ago
ThibG cb71c95e22 Export and import show_reblogs together with following list (#10495)
* Refactor imports

* Export show_reblogs when exporting list of followed users

* Add support for importing show_reblogs with following collection

* Fix tests
6 years ago
Eugen Rochko 67b3b62b98
Improve blocked view of profiles (#10491)
* Revert "Fix filtering of favourited_by, reblogged_by, followers and following (#10447)"

This reverts commit 120544067f.

* Revert "Hide blocking accounts from blocked users (#10442)"

This reverts commit 62bafa20a1.

* Improve blocked view of profiles

- Change "You are blocked" to "Profile unavailable"
- Hide following/followers in API when blocked
- Disable follow button and show "Profile unavailable" on public profile as well
6 years ago
Eugen Rochko e1d0390e29
Add batch actions for approving and rejecting pending accounts (#10469) 6 years ago
Eugen Rochko 2c63e0292a
Fix admin validation being too strict about usernames (#10449)
* Fix admin validation being too strict about usernames

Fix #10446

* Strip Setting.site_contact_username consistently throughout the codebase
6 years ago
ThibG d4882aa64a Export and import `hide_notifications` alongside user mutes (#10335)
* Export hide_notifications along with user mutes

* Import hide_notifications along with muted users list

* Add headers for CSV exports
6 years ago
ThibG 120544067f Fix filtering of favourited_by, reblogged_by, followers and following (#10447) 6 years ago
ThibG 62bafa20a1 Hide blocking accounts from blocked users (#10442)
* Revert "Add indication that you have been blocked in web UI (#10420)"

This reverts commit bd02ec6daa.

* Revert "Add `blocked_by` relationship to the REST API (#10373)"

This reverts commit 9745de883b.

* Hide blocking accounts from search results

* Filter blocking accouts from account followers

* Filter blocking accouts from account's following accounts

* Filter blocking accounts from “reblogged by” and “favourited by” lists

* Remove blocking account from URL search

* Return 410 on trying to fetch user data from a user who blocked us

* Return 410 in /api/v1/account/statuses for suspended or blocking accounts

* Fix status filtering when performing URL search

* Restore some React improvements

Restore some cleanup from bd02ec6daa

* Refactor by adding `without_blocking` scope
6 years ago
slice 85973f4f37 Improvements to image upload validation and creation (#10431)
* Check if image value is nil? before creating an image

Check if uploaded images aren't nil before creating SiteUpload models
for them.

* Validate presence of file in SiteUpload

* Fix file presence validation

* Fabricate SiteUpload#file

* Add link to Creative Commons license
6 years ago
slice 59cce07601 Add thumbnail, hero, and mascot back to Form::AdminSettings::KEYS (#10430)
These will allow these images to be uploaded again on the admin settings
page.
6 years ago
Alex Gessner 69141dca26 squashed identity proof updates (#10375) 6 years ago
Eugen Rochko f1bc90ab50
Rename :poll to :preloadable_poll and :owned_poll to :poll on Status (#10401)
Also, fix some n+1 queries

Resolve #10365
6 years ago
Eugen Rochko e117964325
Change icons of features on admin dashboard to remove bias (#10366)
Red crosses implied that it was bad/unexpected that certain features
were not enabled. In reality, they are options, so showing a green
or grey power-off icon is more appropriate.

Add status of timeline preview as well

Fix sample accounts changing too frequently due to wrong query

Sample accounts are intended to be sorted by popularity
6 years ago
Eugen Rochko 94e5e834f2
Improve performance of list of blocked domains by caching counts (#10374) 6 years ago
Eugen Rochko 45b849bad9
Fix MergeWorker being queued for remote users (#10355) 6 years ago
Eugen Rochko 555c4e11ba
Add validations to admin settings (#10348)
* Add validations to admin settings

- Validate correct HTML markup
- Validate presence of contact username & e-mail
- Validate that all usernames are valid
- Validate that enums have expected values

* Fix code style issue

* Fix tests
6 years ago
Eugen Rochko 00d988955f
If registrations have been re-opened when user confirms account, approve (#10349) 6 years ago
ThibG 80f0910e21 Add support for custom emojis in poll options (#10322)
* Backend changes for custom emoji support in poll options

* Serialize poll emojis in REST API

* Render custom emojis in poll options

* Render custom emoji in poll options on public pages
6 years ago