mastodon

Commit Graph

Author	SHA1	Message	Date
Eugen Rochko	9e3a6d6784	Log BackupWorker backtrace, delete Backup if retries exhausted (#6769 )	7 years ago
Eugen Rochko	9110db41c5	Federate pinned statuses over ActivityPub (#6610 ) * Federate pinned statuses over ActivityPub * Display pinned toots in web UI Fix #6117 * Fix migration * Fix tests * Update outbox_serializer.rb * Update remove_serializer.rb * Update add_serializer.rb * Update fetch_featured_collection_service.rb	7 years ago
Eugen Rochko	61ed133fea	Account archive download (#6460 ) * Fix #201: Account archive download * Export actor and private key in the archive * Optimize BackupService - Add conversation to cached associations of status, because somehow it was forgotten and is source of N+1 queries - Explicitly call GC between batches of records being fetched (Model class allocations are the worst offender) - Stream media files into the tar in 1MB chunks (Do not allocate media file (up to 8MB) as string into memory) - Use #bytesize instead of #size to calculate file size for JSON (Fix FileOverflow error) - Segment media into subfolders by status ID because apparently GIF-to-MP4 media are all named "media.mp4" for some reason * Keep uniquely generated filename in Paperclip::GifTranscoder * Ensure dumped files do not overwrite each other by maintaing directory partitions * Give tar archives a good name * Add scheduler to remove week-old backups * Fix code style issue	7 years ago
Akihiko Odaki	613e7c7521	Rename ResolveRemoteAccountService to ResolveAccountService (#6327 ) The service used to be named ResolveRemoteAccountService resolves local accounts as well.	7 years ago
ThibG	a0de3222dd	Retry delivering toots over ActivityPub for about 2 days (#6298 ) Currently, Mastodon will retry delivering toots for a bit over 1 hour. This is a very short timespan when considering private and direct toots, which cannot be seen by the recipient at all after the delivery attempts have failed. Ideally, private and direct toots should have a different number of retries, but I do not know how to do that.	7 years ago
Eugen Rochko	7badad7797	Fix home regeneration (#6251 ) * Fix regeneration marker not being removed after completion * Return HTTP 206 from /api/v1/timelines/home if regeneration in progress Prioritize RegenerationWorker by putting it into default queue * Display loading indicator and poll home timeline while it regenerates * Add graphic to regeneration message * Make "not found" indicator consistent with home regeneration	7 years ago
Eugen Rochko	ed867eca9d	Move e-mail digest task to sidekiq, reduce workload, improve hint (#6252 )	7 years ago
ThibG	10f6793fd0	Fix PuSH workers (#6200 )	7 years ago
Eugen Rochko	d68868ca14	Lists redis clean-up (#5886 ) * When list is deleted, remove feed from redis * Clean up list feeds of inactive users	7 years ago
Eugen Rochko	85e97ecab6	Fix too many forwards (#5854 ) * Avoid sending explicit Undo->Announce when original deleted * Do not forward a reply back to the server that sent it * Deduplicate inboxes of rebloggers' followers for delete forwarding * Adjust test * Fix wrong class, bad SQL, wrong variable, outdated comment	7 years ago
Eugen Rochko	24cafd73a2	Lists (#5703 ) * Add structure for lists * Add list timeline streaming API * Add list APIs, bind list-account relation to follow relation * Add API for adding/removing accounts from lists * Add pagination to lists API * Add pagination to list accounts API * Adjust scopes for new APIs - Creating and modifying lists merely requires "write" scope - Fetching information about lists merely requires "read" scope * Add test for wrong user context on list timeline * Clean up tests	7 years ago
ThibG	2b1190065c	Retry thread resolving (#5599 ) Thread resolving is one of the few tasks that isn't retried on failure. One common cause for failure of this task is a well-connected user replying to a toot from a little-connected user on a small instance: the small instance will get many requests at once, and will often fail to answer requests within the 10 seconds timeout used by Mastodon. This changes makes the ThreadResolveWorker retry a few times, with a rapidly-increasing time before retries and large random contribution in order to spread the load over time.	7 years ago
Eugen Rochko	1032f3994f	Add ability to disable login and mark accounts as memorial (#5615 ) Fix #5597	7 years ago
aschmitz	554c2fd8af	Clean up reblog tracking keys, related improvements (#5428 ) * Clean up reblog-tracking sets from FeedManager Builds on #5419, with a few minor optimizations and cleanup of sets after they are no longer needed. * Update tests, fix multiply-reblogged case Previously, we would have lost the fact that a given status was reblogged if the displayed reblog of it was removed, now we don't. Also added tests to make sure FeedManager#trim cleans up our reblog tracking keys, fixed up FeedCleanupScheduler to use the right loop, and fixed the test for it.	7 years ago
Eugen Rochko	34118169ac	Keep references to all reblogs of a status on home feed (#5419 ) * Keep references to all reblogs of a status on home feed When inserting reblog: Add to set of reblogs of this status on the feed, if original status was present in the feed, add it to that set as well. When removing a reblog: Remove it from that set. Take random remaining item from the set. If one exists, re-insert it into feed, otherwise do not re-insert anything. Fix #4210 * When original is removed, toss out reblog references	7 years ago
abcang	2eab41cd1a	Close connection when succeeded posting (#5390 ) * Close connection when succeeded posting * Update webmock	7 years ago
Eugen Rochko	0717d9b3e6	Set snowflake IDs for backdated statuses (#5260 ) - Rename Mastodon::TimestampIds into Mastodon::Snowflake for clarity - Skip for statuses coming from inbox, aka delivered in real-time - Skip for statuses that claim to be from the future	7 years ago
unarist	5c8ca024ef	Improve error handling on LinkCrawlWorker (#5250 ) * Improve error handling on LinkCrawlWorker * Ignore TimeoutError and InvalidURIError too * Record errors to debug log * Enable dead job queue on LinkCrawlWorker Since most of acceptable errors were already ignored, only our side issue should go to dead job queue. * Ignore all http gem errors	7 years ago
Eugen Rochko	cdd5ef691b	Use separate workers to process imports, retry failures (#5207 )	7 years ago
ThibG	f7c909e290	Retry ActivityPub delivery a few more times (#5014 )	7 years ago
Eugen Rochko	f4ca116ea8	After 7 days of repeated delivery failures, give up on inbox (#5131 ) - A successful delivery cancels it out - An incoming delivery from account of the inbox cancels it out	7 years ago
abcang	3d9b8847d2	Flush body when POST requests (#5128 )	7 years ago
Eugen Rochko	e528114c53	Follow-up to #4582 and #5027 , removing dead code (#5101 )	7 years ago
Daigo 3 Dango	a0bbeafb04	Suppress backtrace when failed to communicate with a remote instance (#5076 )	7 years ago
unarist	0698c610a6	Fix an error in ReplyDistributionWorker when replied status was deleted (#4974 ) Reply distribution is proceed by Sidekiq, so replied status may be deleted before this.	8 years ago
sdukhovni	67559361e8	Add scheduled worker to purge old user IPs (#4951 ) * Add scheduled worker to purge old user IPs * Use ruby 1.9 hash syntax	8 years ago
abcang	1aad015bbb	Revert unique retry job (#4937 ) * Revert "Enable UniqueRetryJobMiddleware even when called from sidekiq worker (#4836)" This reverts commit `6859d4c028`. * Revert "Do not execute the job with the same arguments as the retry job (#4814)" This reverts commit `be7ffa2d75`.	8 years ago
ThibG	af00220d79	Fix refollowing (#4931 ) * Make RefollowWorker ActivityPub-only to avoid potential identifier mismatches * Don't call RefollowWorker on new accounts	8 years ago
ThibG	f29918e707	[WiP] Whenever a remote keypair changes, unfollow them and re-subscribe to … (#4907 ) * Whenever a remote keypair changes, unfollow them and re-subscribe to them In Mastodon (it could be different for other OStatus or AP-enabled software), a keypair change is indicative of whole user (or instance) data loss. In this situation, the “new” user might be different, and almost certainly has an empty followers list. In this case, Mastodon instances will disagree on follower lists, leading to unreliable delivery and “shadow followers”, that is users believed by a remote instance to be followers, without the affected user knowing. Drawbacks of this change are: 1. If an user legitimately changes public key for some reason without losing data (not possible in Mastodon at the moment), they will have their remote followers unsubscribed/re-subscribed needlessly. 2. Depending of the number of remote followers, this may generate quite some traffic. 3. If the user change is an attempt at usurpation, the remote followers will unknowingly follow the usurper. Note that this is not a change of behavior, Mastodon already behaves like that, although delivery might be unreliable, and the usurper would not have known the former user's followers. * Rename ResubscribeWorker to RefollowWorker * Process followers in batches	8 years ago
Eugen Rochko	7d7844a47f	Default follows for new users (#4871 ) When a new user confirms their e-mail, bootstrap their home timeline by automatically following a set of accounts. By default, all local admin accounts (that are unlocked). Can be customized by new admin setting (comma-separated usernames, local and unlocked only)	8 years ago
Eugen Rochko	e7adbf572a	Switch to static URIs, new URI format in both protocols for new statuses (#4815 ) * Decouple Status#local? from uri being nil * Replace on-the-fly URI generation with stored URIs - Generate URI in after_save hook for local statuses - Use static value in TagManager when available, fallback to tag format - Make TagManager use ActivityPub::TagManager to understand new format - Adjust tests * Use other heuristic for locality of old statuses, do not perform long query * Exclude tombstone stream entries from Atom feed * Prevent nil statuses from landing in Pubsubhubbub::DistributionWorker * Fix URI not being saved (#4818) * Add more specs for Status * Save generated uri immediately and also fix method order to minimize diff. * Fix alternate HTML URL in Atom * Fix tests * Remove not-null constraint from statuses migration to speed it up	8 years ago
abcang	be7ffa2d75	Do not execute the job with the same arguments as the retry job (#4814 )	8 years ago
Eugen Rochko	a187dcefa1	Instantly upgrade account to ActivityPub if we receive ActivityPub payload (#4766 )	8 years ago
Eugen Rochko	7b8f262840	Forward ActivityPub creates that reply to local statuses (#4709 ) * Forward ActivityPub creates that reply to local statuses * Fix test * Fix wrong signers	8 years ago
Eugen Rochko	4c76402ba1	Serialize ActivityPub alternate link into OStatus deletes, handle it (#4730 ) Requires moving Atom rendering from DistributionWorker (where `stream_entry.status` is already nil) to inline (where `stream_entry.status.destroyed?` is true) and distributing that. Unfortunately, such XML renderings can no longer be easily chained together into one payload of n items.	8 years ago
Eugen Rochko	0397c58b61	Forward ActivityPub deletes to followers of rebloggers (#4706 )	8 years ago
Eugen Rochko	00840f4f2e	Add handling of Linked Data Signatures in payloads (#4687 ) * Add handling of Linked Data Signatures in payloads * Add a way to sign JSON, fix canonicalization of signature options * Fix signatureValue encoding, send out signed JSON when distributing * Add missing security context	8 years ago
masarakki	649a20ab46	authorize-follow-requests-after-unlocking (#4658 )	8 years ago
Eugen Rochko	cf615abbf9	Add configuration to disable private status federation over PuSH (#4582 )	8 years ago
Eugen Rochko	f391a4673a	Periodically remove expired PuSH subscribers (#4654 )	8 years ago
Eugen Rochko	6e9eda5331	ActivityPub migration procedure (#4617 ) * ActivityPub migration procedure Once one account is detected as going from OStatus to ActivityPub, invalidate WebFinger cache for other accounts from the same domain * Unsubscribe from PuSH updates once we receive an ActivityPub payload * Re-subscribe to PuSH unless already unsubscribed, regardless of protocol	8 years ago
Eugen Rochko	412ea87306	Improve ActivityPub/OStatus compatibility (#4632 ) Note: OStatus URIs are invalid for ActivityPub. But we have them for as long as we want to keep old OStatus-sourced content and as long as we remain OStatus-compatible. - In Announce handling, if object URI is not a URL, fallback to object URL - Do not use specialized ThreadResolveWorker, rely on generalized handling - When serializing notes, if parent's URI is not a URL, use parent's URL	8 years ago
Eugen Rochko	b7370ac8ba	ActivityPub delivery (#4566 ) * Deliver ActivityPub Like * Deliver ActivityPub Undo-Like * Deliver ActivityPub Create/Announce activities * Deliver ActivityPub creates from mentions * Deliver ActivityPub Block/Undo-Block * Deliver ActivityPub Accept/Reject-Follow * Deliver ActivityPub Undo-Follow * Deliver ActivityPub Follow * Deliver ActivityPub Delete activities Incidentally fix #889 * Adjust BatchedRemoveStatusService for ActivityPub * Add tests for ActivityPub workers * Add tests for FollowService * Add tests for FavouriteService, UnfollowService and PostStatusService * Add tests for ReblogService, BlockService, UnblockService, ProcessMentionsService * Add tests for AuthorizeFollowService, RejectFollowService, RemoveStatusService * Add tests for BatchedRemoveStatusService * Deliver updates to a local account to ActivityPub followers * Minor adjustments	8 years ago
Eugen Rochko	3d47154c20	Only PuSH-resubscribe to OStatus accounts (#4583 )	8 years ago
Eugen Rochko	dd7ef0dc41	Add ActivityPub inbox (#4216 ) * Add ActivityPub inbox * Handle ActivityPub deletes * Handle ActivityPub creates * Handle ActivityPub announces * Stubs for handling all activities that need to be handled * Add ActivityPub actor resolving * Handle conversation URI passing in ActivityPub * Handle content language in ActivityPub * Send accept header when fetching actor, handle JSON parse errors * Test for ActivityPub::FetchRemoteAccountService * Handle public key and icon/image when embedded/as array/as resolvable URI * Implement ActivityPub::FetchRemoteStatusService * Add stubs for more interactions * Undo activities implemented * Handle out of order activities * Hook up ActivityPub to ResolveRemoteAccountService, handle Update Account activities * Add fragment IDs to all transient activity serializers * Add tests and fixes * Add stubs for missing tests * Add more tests * Add more tests	8 years ago
Yamagishi Kazutoshi	3e7a541e09	Change RuboCop rules to loose (#4464 )	8 years ago
Eugen Rochko	4e2f2fab73	Fix guard clause in WebPushNotificationWorker (#4421 )	8 years ago
Eugen Rochko	6e186b9c77	When PuSH subscribe attempts are exhausted, unsubscribe (#4422 )	8 years ago
Clworld	994d948c39	Add callback_url/acct information for Sidekiq PuSH workers Exception. (#4281 ) * Add destination informations to exception on SubscribeWorker and DeliveryWorker. * Simplify delivery error message. * Prevent changing Exception type... * fix typo.	8 years ago
Sorin Davidoi	8b43d6bf9c	fix(web_push_notification_worker): Guard against deleted notifications (#4379 )	8 years ago

1 2 3 4

152 Commits (3b440bd5af92682c388140035c1bcdc6f9711b1d)