769 Commits (ac3c83ef6fe207a22d67ff8912e74c21c6b61daf)

Author	SHA1	Message	Date
Eugen Rochko	368a87755b	Fix account migration not affecting followers on origin server (#11980)	5 years ago
Eugen Rochko	3ec80c7aec	Fix preview card image not being re-fetched even if link is re-posted (#11981) ... Fix #11956	5 years ago
Eugen Rochko	ab33c4df94	Add `exclude_unreviewed` param to `GET /api/v2/search` REST API (#11977) ... Make it so normal search returns even unreviewed matches, but autosuggestions do not. Fix #11960	5 years ago
ThibG	18b451c0e6	Change silences to always require approval on follow (#11975) ... * Change silenced accounts to require approval on follow * Also require approval for follows by people explicitly muted by target accounts * Do not auto-accept silenced or muted accounts when switching from locked to unlocked * Add `follow_requests_count` to verify_credentials * Show “Follow requests” menu item if needed even if account is locked * Add tests * Correctly reflect that follow requests weren't auto-accepted when local account is silenced * Accept follow requests from user-muted accounts to avoid leaking mutes	5 years ago
Eugen Rochko	2f90a38f44	Fix unreviewed hashtag not being found by exact case-insensitive match (#11976)	5 years ago
Jeong Arm	00d7bdcc2a	Fix search error when ElasticSearch is enabled but not available (#11954) ... * Fallback to Database search when ES not available * Prevent double work if ES gives 0 result * Apply suggestion from code review	5 years ago
Eugen Rochko	4f6af87906	Change spam check to apply to local accounts and add a threshold (#11806) ... Instead of detecting spam on first duplicate message, add a threshold of 5 such messages to reduce false positives	6 years ago
Eugen Rochko	38dc51b2d6	Fix Move handler queuing jobs that will fail if account is suspended (#11864) ... Don't put Move handler on cooldown if it didn't run. Skip unmerging from timelines to save unnecessary work.	6 years ago
Eugen Rochko	50edf84898	Fix search API not resolving URL when type is given (#11822) ... Fix #11771	6 years ago
Eugen Rochko	c5d37f18cb	Change deletes to preserve soft-deleted statuses in unresolved reports (#11805) ... Change all account actions except "none" to resolve all unresolved reports Refactor `SuspendAccountService` to be more readable	6 years ago
Eugen Rochko	43f56f1291	Change account deletion page to have better explanations (#11753) ... Fix deletion of unconfirmed account not freeing up the username Add prefill of logged-in user's email in the reconfirmation form	6 years ago
Eugen Rochko	cb447b28c4	Add profile directory to web UI (#11688) ... * Add profile directory to web UI * Add a line of bio to the directory	6 years ago
Eugen Rochko	97192d9a77	Fix remote and staff-removed statuses leaving media behind for a day (#11638) ... The reason for unattaching media instead of removing it is to support delete & redraft functionality, but remote or staff-removed statuses will never be redrafted, so the media should be deleted immediately	6 years ago
Eugen Rochko	cc0a55cf9a	Add more accurate hashtag search (#11579) ... * Add more accurate hashtag search Using ElasticSearch to index hashtags with edge n-grams and score them by usage within the last 7 days since last activity. Only hashtags that have been reviewed and are listable can appear in searches, unless they match the query exactly * Fix search analyzer dropping non-ascii characters	6 years ago
Eugen Rochko	70da6d6630	Fix accounts search by full/partial display name and others (#11580) ... - Restrict followers counts to local users to minimize local advantage - Fix emoji shortcodes causing error in search - Fix search syntax parse errors not being caught	6 years ago
Eugen Rochko	8fdff2748f	Add more accurate account search (#11537) ... * Add more accurate account search When ElasticSearch is available, a more accurate search is implemented: - Using edge n-gram index for acct and display name - Using asciifolding and cjk width normalization on display names - Using Gaussian decay on account activity for additional scoring (recency) - Using followers/friends ratio for additional scoring (spamminess) - Using followers number for additional scoring (size) The exact match precedence only takes effect when the input conforms to the username format and the username part of it is complete, i.e. when the user started typing the domain part. * Support single-letter usernames * Fix tests * Fix not picking up account updates * Add weights and normalization for scores, skip zero terms queries * Use local counts for accounts index, adjust search parameters * Fix mistakes * Using updated_at of accounts is inadequate for remote accounts	6 years ago
Eugen Rochko	c5661b0f92	Fix account search always returning exact match on paginated results (#11525) ... Fix #11365	6 years ago
Eugen Rochko	f51c7c105f	Fix acct URIs with IDN domains not being resolved (#11520) ... Fix #11494	6 years ago
ThibG	bced70469a	Add domain block notes (#11515) ... * Add database columns for adding notes to domain blocks/restrctions * Add admin UI to set private and public comments when blocking a domain * Add text for private and public comments on domain blocks * Show domain block comments in admin UI * Add comments to the domain block undo page * Make UnblockDomainService more robust regarding upgraded domain blocks * Allow editing domain blocks * Rename button from “undo domain block” to “view domain block” in account admin UI * Change test to unsilence silenced users from upgraded blocks	6 years ago
Eugen Rochko	5e35aa8280	Fix non-lowercase hashtags not being picked up by the streaming API (#11508) ... Regression from f371b32 Fix hashtag links always being lowercase	6 years ago
Eugen Rochko	24552b5160	Add whitelist mode (#11291)	6 years ago
Eugen Rochko	f371b32137	Change hashtags to preserve first-used casing (#11416)	6 years ago
Eugen Rochko	b9fbcbfe4e	Add search syntax for operators and phrases (#11411)	6 years ago
ThibG	92569ffde8	Fix invites not being disabled upon account suspension (#11412) ... * Disable invite links from disabled/suspended users * Add has_many invites relationship to users * Destroy unused invites when suspending an account	6 years ago
Eugen Rochko	4eeff26533	Change account domain block to clear out notifications and follows (#11393)	6 years ago
Eugen Rochko	bd87e66679	Remove WebSub subscriptions (#11303)	6 years ago
ThibG	2f813b7ea1	Disable LDSigning when AUTHORIZED_FETCH is set to true (#11295)	6 years ago
ThibG	3595ce6325	Fix leaking private statuses the admin account follows (#11300) ... Now that the request is signed, it can return private toots. Do not leak them.	6 years ago
Eugen Rochko	6ff67be0f6	Add a spam check (#11217) ... * Add a spam check * Use Nilsimsa to generate locality-sensitive hashes and compare using Levenshtein distance * Add more tests * Add exemption when the message is a reply to something that mentions the sender * Use Nilsimsa Compare Value instead of Levenshtein distance * Use MD5 for messages shorter than 10 characters * Add message to automated report, do not add non-public statuses to automated report, add trust level to accounts and make unsilencing raise the trust level to prevent repeated spam checks on that account * Expire spam check data after 3 months * Add support for local statuses, reduce expiration to 1 week, always create a report * Add content warnings to the spam check and exempt empty statuses * Change Nilsimsa threshold to 95 and make sure removed statuses are removed from the spam check * Add all matched statuses into automatic report	6 years ago
ThibG	4e1260feaa	Fix BlockService trying to reject incorrect follow request (#11288) ... Fixes #11148	6 years ago
Eugen Rochko	4e8dcc5dbb	Add HTTP signatures to all outgoing ActivityPub GET requests (#11284)	6 years ago
Eugen Rochko	5d3feed191	Refactor fetching of remote resources (#11251)	6 years ago
ThibG	85eb418e1f	Fix handling of webfinger redirects in ResolveAccountService (#11279)	6 years ago
Eugen Rochko	4e92183227	Refactor domain block checks (#11268)	6 years ago
Eugen Rochko	63c7fe8e48	Refactor controllers for statuses, accounts, and more (#11249)	6 years ago
ThibG	87c1ad4ea3	Fix BackupService crashing when an attachment is missing (#11241) ... * Fix BackupService crashing when an attachment is missing For various reasons such as admin error or out-of-sync media and database backups, it might be possible for local attachments to be lost. This commit allows the BackupService to continue its work even if some media file is missing. * Change error message	6 years ago
Eugen Rochko	b851456139	Remove Atom feeds and old URLs in the form of `GET /:username/updates/:id` (#11247)	6 years ago
Eugen Rochko	23aeef52cc	Remove Salmon and PubSubHubbub (#11205) ... * Remove Salmon and PubSubHubbub endpoints * Add error when trying to follow OStatus accounts * Fix new accounts not being created in ResolveAccountService	6 years ago
Eugen Rochko	2cfa427ea7	Change domain block behaviour to prevent creation of accounts from suspended domains (#11219)	6 years ago
ThibG	915c619394	Add support for Audio activities (#11189) ... Fixes #11127	6 years ago
Eugen Rochko	707ddf7808	Change domain blocks to automatically support subdomains (#11138) ... * Change domain blocks to automatically support subdomains If a more authoritative domain is blocked (example.com), then the same block will be applied to a subdomain (foo.example.com) * Match subdomains of existing accounts when blocking/unblocking domains * Improve code style	6 years ago
Eugen Rochko	f7f23b4a19	Add audio uploads (#11123) ... * Add audio uploads Fix #4827 Accept uploads of OGG, WAV, FLAC, OPUS and MP3 files, and converts them to OGG. Media attachments get a new `audio` type. In the UI, audio uploads are displayed identically to video uploads. * Improve code style	6 years ago
Eugen Rochko	f2b743e715	Refactor all ActivityPub deliveries to be serialized and signed through one concern (#10966)	6 years ago
ThibG	a1519a8ef5	Prevent from publicly boosting one's own private toots (#10775)	6 years ago
ThibG	14f6ce2885	Record account suspend/silence time and keep track of domain blocks (#10660) ... * Record account suspend/silence time and keep track of domain blocks * Also unblock users who were suspended/silenced before dates were recorded * Add tests * Keep track of suspending date for users suspended through the CLI * Show accurate number of accounts that would be affected by unsuspending an instance * Change migration to set silenced_at and suspended_at * Revert "Also unblock users who were suspended/silenced before dates were recorded" This reverts commit a015c65d2d. * Switch from using suspended and silenced to suspended_at and silenced_at * Add post-deployment migration script to remove `suspended` and `silenced` columns * Use Account#silence! and Account#suspend! instead of updating the underlying property * Add silenced_at and suspended_at migration to post-migration * Change account fabricator to translate suspended and silenced attributes * Minor fixes * Make unblocking domains always retroactive	6 years ago
ThibG	b1a0322a06	Reject follow requests of blocked users (#10633)	6 years ago
Daniel Aleksandersen	10bdd912d6	Treat meta[property] as a space-separated list (#10604) ... The @property attribute in HTML is a space-separated list of values. This change normalizes whitespace and finds the desired value in the list instead of requiring an exact single-value match. More details: https://www.ctrl.blog/entry/rdfa-socialmedia-metadata.html	6 years ago
ThibG	cb71c95e22	Export and import show_reblogs together with following list (#10495) ... * Refactor imports * Export show_reblogs when exporting list of followed users * Add support for importing show_reblogs with following collection * Fix tests	6 years ago
Eugen Rochko	67b3b62b98	Improve blocked view of profiles (#10491) ... * Revert "Fix filtering of favourited_by, reblogged_by, followers and following (#10447)" This reverts commit 120544067f. * Revert "Hide blocking accounts from blocked users (#10442)" This reverts commit 62bafa20a1. * Improve blocked view of profiles - Change "You are blocked" to "Profile unavailable" - Hide following/followers in API when blocked - Disable follow button and show "Profile unavailable" on public profile as well	6 years ago
ThibG	d4882aa64a	Export and import `hide_notifications` alongside user mutes (#10335) ... * Export hide_notifications along with user mutes * Import hide_notifications along with muted users list * Add headers for CSV exports	6 years ago
ThibG	62bafa20a1	Hide blocking accounts from blocked users (#10442) ... * Revert "Add indication that you have been blocked in web UI (#10420)" This reverts commit bd02ec6daa. * Revert "Add `blocked_by` relationship to the REST API (#10373)" This reverts commit 9745de883b. * Hide blocking accounts from search results * Filter blocking accouts from account followers * Filter blocking accouts from account's following accounts * Filter blocking accounts from “reblogged by” and “favourited by” lists * Remove blocking account from URL search * Return 410 on trying to fetch user data from a user who blocked us * Return 410 in /api/v1/account/statuses for suspended or blocking accounts * Fix status filtering when performing URL search * Restore some React improvements Restore some cleanup from bd02ec6daa * Refactor by adding `without_blocking` scope	6 years ago
Eugen Rochko	1714ea5978	Add ActivityPub representation for identity proofs (#10414) ... * Add ActivityPub representation for identity proofs * Add tests	6 years ago
Eugen Rochko	f1bc90ab50	Rename :poll to :preloadable_poll and :owned_poll to :poll on Status (#10401) ... Also, fix some n+1 queries Resolve #10365	6 years ago
Eugen Rochko	9d0573f574	Fix vote-after-expiration not showing error, count vote as interaction (#10352)	6 years ago
ThibG	62dd0b7f9c	Fix processing poll Updates (#10333) ... ActivityPub::ProcessPollService was checking the JSON-LD context although it was passed only the `Question` object embedded in the `Update` activity.	6 years ago
ThibG	1217185cfa	Do not distribute Delete when rejecting unapproved accounts (#10321)	6 years ago
ThibG	a20354a20b	Set and store report URIs (#10303) ... Fixes #10271	6 years ago
ThibG	6b0eda14a1	Avoid race condition when streaming deleted statuses (#10280) ... * Avoid race condition when streaming deleted statuses * Move redis lock to DistributionWorker to avoid extra Redis value	6 years ago
Eugen Rochko	3ad3223b46	Fix detailed poll validation errors not being returned in the API (#10261) ... No more "Owned poll is invalid"	6 years ago
Eugen Rochko	ba84b6d4d7	Add `visibility` param to reblog REST API (#9851) ... Use async worker for creating reblog notification to improve performance	6 years ago
Eugen Rochko	51e154f5e8	Admission-based registrations mode (#10250) ... Fix #6856 Fix #6951	6 years ago
Eugen Rochko	9f5b55ad4f	Fix poll update handler calling method was that was not available (#10246) ... * Fix poll update handler calling method was that was not available Fix regression from #10209 * Refactor VoteService * Refactor ActivityPub::DistributePollUpdateWorker and optimize it * Fix typo * Fix typo	6 years ago
Eugen Rochko	a41afef80c	Optimize RemoveStatusService by removing n+1 query and using shared inboxes (#10247)	6 years ago
ThibG	3a92885a86	Support pushing and receiving updates to poll tallies (#10209) ... * Process incoming poll tallies update * Send Update on poll vote * Do not send Updates for a poll more often than once every 3 minutes * Include voters in people to notify of results update * Schedule closing poll worker on poll creation * Add new notification type for ending polls * Add front-end support for ended poll notifications * Fix UpdatePollSerializer * Fix Updates not being triggered by local votes * Fix tests failure * Fix web push notifications for closing polls * Minor cleanup * Notify voters of both remote and local polls when those close * Fix delivery of poll updates to mentioned accounts and voters	6 years ago
ThibG	c11dff5049	Reject existing Follows when suspending a remote account (#10230) ... * Reject existing Follows when suspending a remote account Partial fix to #10229 * Add tests	6 years ago
ThibG	3aaac4f134	Do not allow adding votes to expired polls (#10214) ... * Do not allow adding votes to expired polls * Only validate expires_at on create	6 years ago
ThibG	09c042aa10	Handle StaleObjectError when retrieving polls (#10208)	6 years ago
ThibG	df5924a1db	Do not error out on unsalvageable errors in FetchRepliesService (#10175) ... * Do not error out on unsalvageable errors in FetchRepliesService Fixes #10152 * Fix FetchRepliesWorker erroring out on deleted statuses	6 years ago
ThibG	d785497ba5	Fix suspended account's fields being set as empty dict instead of list (#10178) ... Fixes #10177	6 years ago
Eugen Rochko	4037b5eb1e	Fix last_fetched_at not being set on polls (#10170)	6 years ago
Eugen Rochko	e6900b167b	Fix another typo in ActivityPub::FetchRemotePollService (#10146)	6 years ago
Eugen Rochko	878a75ba21	Fix typo in ActivityPub::FetchRemotePollService (#10145)	6 years ago
Eugen Rochko	ae1b9cf70a	Fix remote poll expiration time (#10144)	6 years ago
ThibG	f821eca3b3	Correctly make polls and media mutually exclusive (#10141)	6 years ago
ThibG	e13d3792f3	Make sure the poll is created before storing its id (#10142) ... * Make sure the poll is created before storing its id * Fix updating poll results * Support fetching Question activities from the search bar	6 years ago
Eugen Rochko	230a012f00	Add polls (#10111) ... * Add polls Fix #1629 * Add tests * Fixes * Change API for creating polls * Use name instead of content for votes * Remove poll validation for remote polls * Add polls to public pages * When updating the poll, update options just in case they were changed * Fix public pages showing both poll and other media	6 years ago
ThibG	9d3c6f1849	Improved remote thread fetching (#10106) ... * Fetch up to 5 replies when discovering a new remote status This is used for resolving threads downwards. The originating server must add a “replies” attributes with such replies for it to be useful. * Add some tests for ActivityPub::FetchRepliesWorker * Add specs for ActivityPub::FetchRepliesService * Serialize up to 5 public self-replies for ActivityPub notes * Add specs for ActivityPub::NoteSerializer * Move exponential backoff logic to a worker concern * Fetch first page of paginated collections when fetching thread replies * Add specs for paginated collections in replies * Move Note replies serialization to a first CollectionPage The collection isn't actually paginable yet as it has no id nor a `next` field. This may come in another PR. * Use pluck(:uri) instead of map(&:uri) to improve performances * Fix fetching replies when they are in a CollectionPage	6 years ago
Eugen Rochko	e7f20cc43f	Add type, limit, offset, min_id, max_id, account_id to search API (#10091) ... * Add type, limit, offset, min_id, max_id, account_id to search API Fix #8939 * Make the offset work on accounts and hashtags search as well * Assure brakeman we are not doing mass assignment here * Do not allow paginating unless a type is chosen * Fix search query and index id field on statuses instead of created_at	6 years ago
NOGISAKA Sadata	e050958794	Skip some methods in BatchedRemoveStatusService when account is nil (#10095) ... * Skip some methods in BatchedRemoveStatusService when account is nil Sometimes `bin/tootctl accounts cull` fails with below error: undefined method `followers_for_local_distribution' for nil:NilClass (NoMethodError) This commit makes BatchedRemoveStatusService to skip below methods when `account` is nil: - unpush_from_home_timelines() - unpush_from_list_timelines() - batch_stream_entries() * Fix rubocop error: Use `next` to skip iteration.	6 years ago
Eugen Rochko	dad339da6d	Filter incoming Create activities by relation to local activity (#10005) ... Reject those from accounts with no local followers, from relays that are not enabled, which do not address local accounts and are not replies to accounts that do have local followers	6 years ago
ThibG	99c4b5b75a	Move sending account Delete to anyone but the account's followers to the pull̀ queue (#10016)	6 years ago
abcang	e186bd2fb0	Fix Tombstone.delete_all ArgumentError (#9978)	6 years ago
Eugen Rochko	364f2ff9aa	Add featured hashtags to profiles (#9755) ... * Add hashtag filter to profiles GET /@:username/tagged/:hashtag GET /api/v1/accounts/:id/statuses?tagged=:hashtag * Display featured hashtags on public profile * Use separate model for featured tags * Update featured hashtag counters on-write * Limit featured tags to 10	6 years ago
Eugen Rochko	d14c276e58	Add option to overwrite imported data (#9962) ... * Add option to overwrite imported data Fix #7465 * Add import for domain blocks	6 years ago
ysksn	bcfff65195	Create Redisable#redis (#9633) ... * Create Redisable * Use #redis instead of Redis.current	6 years ago
ThibG	e2a5be6e9a	Prevent posting toots with media attachments from someone else (#9921)	6 years ago
Renato "Lond" Cerqueira	e1ec3a9f09	Rescue SSL error in verify link service (#9914) ... If the first link to be verified contains a rel=me link with a SSL error, the VerifyAccountLinksWorker will fail and not try the following links. This rescues the SSL error when fetching the link, avoiding this issue.	6 years ago
ThibG	061feb63ed	Fix scheduled toot with media immediately creating a toot (#9894) ... * Add test for not persisting status when attaching media to scheduled toot * Prevent status used for validation from being persisted to the database Fixes #9893 Thanks to tateisu for the help investigating this.	6 years ago
Eugen Rochko	6ca9015789	Fix error when `scheduled_at` attribute is malformed (#9866)	6 years ago
ThibG	aeb124491d	Reject existing Follow in addition to sending a Block (#9811) ... Mastodon expects remote servers to remove follow relationships upon receiving a Block. However, the spec only evokes Block activities in a C2S context, never in a S2S context. This PR, in addition to federating the Block, explicitly sends a Reject for any affected follow relationship, which makes a bit more sense with regards to the spec.	6 years ago
ThibG	75b1488cf4	Add tombstones for remote statuses (#9830) ... * Add Tombstone model to remember object deletion * Do not recreate a status if it has been deleted * Record Tombstone for remote deleted items Also, only record deleted items from same-host actors * Clear an user's tombstones when their key change	6 years ago
Eugen Rochko	69f782b54d	Fix code style of regeneration-related code (#9843)	6 years ago
Renato "Lond" Cerqueira	5c5e14c816	Fix undefined method error in sidekiq (#9807) ... * Fix undefined method error in sidekiq Body can be not nil but still be empty, which causes a `NoMethodError: undefined method `[]' for nil:NilClass` further in the code. This checks for an empty body to avoid the issue. * Fix codeclimate issue	6 years ago
ThibG	28b482874a	Improvements to signature verification (#9667) ... * Refactor signature verification a bit * Rescue signature verification if recorded public key is invalid Fixes #8822 * Always re-fetch AP signing key when HTTP Signature verification fails But when the account is not marked as stale, avoid fetching collections and media, and avoid webfinger round-trip. * Apply stoplight to key/account update as well as initial key retrieval	6 years ago
Eugen Rochko	43c61bca60	Add locale param to sign-up API (#9747) ... Fix #9627	6 years ago
Eugen Rochko	1cbdf8d218	Fix wrong param name in scheduled statuses and return params in API (#9725) ... The database column and API param are called in_reply_to_id, not in_reply_to_status_id, so it makes no sense to encode it that way	6 years ago
Eugen Rochko	a49d43d112	Add scheduled statuses (#9706) ... Fix #340	6 years ago
Naoki Kosaka	fae32634b1	Use Contact User as Relay, Report, Subscribe. (#9661) ... * Use Contact User as Relay, Report, Subscribe. * Use Account.representative to fetch contact user. * Use find_local. * No reason to use Account.representative in subscribe_service. * Don't required representative! * Fallback is included in Account.representative method.	6 years ago
kodai	6a090db4c9	Fix links not being verified (#9673) ... Fix #9365	6 years ago
Jeong Arm	511231ce57	Skip full text search if failed (#9654) ... * Skip full text search if failed * Move rescue block to "perform_statuses_search!"	6 years ago