mastodon

Commit Graph

Author	SHA1	Message	Date
ThibG	a24605961a	Fixes/do not override timestamps (#7336 ) * Revert "Fixes/do not override timestamps (#7331)" This reverts commit `581a5c9d29`. * Document Snowflake ID corner-case a bit more Snowflake IDs are used for two purposes: making object identifiers harder to guess and ensuring they are in chronological order. For this reason, they are based on the `created_at` attribute of the object. Unfortunately, inserting items with older snowflakes IDs will break the assumption of consumers of the paging APIs that new items will always have a greater identifier than the last seen one. * Add `override_timestamps` virtual attribute to not correlate snowflake ID with created_at	7 years ago
ThibG	581a5c9d29	Fixes/do not override timestamps (#7331 ) * Do not override timestamps for incoming toots * Remove every reference to override_timestamps Statuses are now created with the announced publishing date and are only pushed to timelines if that date is at most 6 hours earlier than the time at which it is processed.	7 years ago
Eugen Rochko	d5fa4fbcd2	Revert "Do not override timestamps for incoming toots" (#7330 ) * Revert "Weblate translations 20180503 (#7325)" This reverts commit `dfa6bccb64`. * Revert "Prevent timeline from moving when cursor is hovering over it (fixes #7278) (#7327)" This reverts commit `58852695c8`. * Revert "Add pry-byebug (#7307)" This reverts commit `ab773e4d5f`. * Revert "Do not override timestamps for incoming toots (#7326)" This reverts commit `bd36791832`.	7 years ago
ThibG	bd36791832	Do not override timestamps for incoming toots (#7326 )	7 years ago
Eugen Rochko	658cbc9425	Improve PostStatusService performance (#7317 ) Offload creation of local notifications to a worker. Remove two redundant SQL queries from ProcessMentionsService, remove n+1 XML/JSON serialization via memoization	7 years ago
Eugen Rochko	cb5b5cb5f7	Slightly reduce RAM usage (#7301 ) * No need to re-require sidekiq plugins, they are required via Gemfile * Add derailed_benchmarks tool, no need to require TTY gems in Gemfile * Replace ruby-oembed with FetchOEmbedService Reduce startup by 45382 allocated objects * Remove preloaded JSON-LD in favour of caching HTTP responses Reduce boot RAM by about 6 MiB * Fix tests * Fix test suite by stubbing out JSON-LD contexts	7 years ago
Eugen Rochko	778562c223	Ensure SynchronizeFeaturedCollectionWorker is unique and clean up (#7043 ) * Ensure SynchronizeFeaturedCollectionWorker is unique and clean up Fix #7041 * Fix code style issue	7 years ago
Eugen Rochko	d4de2239b0	Add a circuit breaker for ActivityPub deliveries (#7053 )	7 years ago
Eugen Rochko	f1867a7388	Adjust privacy policy (#6666 ) * Adjust privacy policy to be more specific to Mastodon Fix #6613 * Change data retention of IP addresses from 5 years to 1 year * Add even more information * Remove all (now invalid) translations of the privacy policy * Add information about archive takeout, remove pointless consent section * Emphasis on DM privacy * Improve wording * Add line about data use for moderation purposes	7 years ago
Akihiko Odaki	40e5d2303b	Validate HTTP response length while receiving (#6891 ) to_s method of HTTP::Response keeps blocking while it receives the whole content, no matter how it is big. This means it may waste time to receive unacceptably large files. It may also consume memory and disk in the process. This solves the inefficency by checking response length while receiving.	7 years ago
Akihiko Odaki	54b273bf99	Close http connection in perform method of Request class (#6889 ) HTTP connections must be explicitly closed in many cases, and letting perform method close connections makes its callers less redundant and prevent them from forgetting to close connections.	7 years ago
Eugen Rochko	9e3a6d6784	Log BackupWorker backtrace, delete Backup if retries exhausted (#6769 )	7 years ago
Eugen Rochko	9110db41c5	Federate pinned statuses over ActivityPub (#6610 ) * Federate pinned statuses over ActivityPub * Display pinned toots in web UI Fix #6117 * Fix migration * Fix tests * Update outbox_serializer.rb * Update remove_serializer.rb * Update add_serializer.rb * Update fetch_featured_collection_service.rb	7 years ago
Eugen Rochko	61ed133fea	Account archive download (#6460 ) * Fix #201: Account archive download * Export actor and private key in the archive * Optimize BackupService - Add conversation to cached associations of status, because somehow it was forgotten and is source of N+1 queries - Explicitly call GC between batches of records being fetched (Model class allocations are the worst offender) - Stream media files into the tar in 1MB chunks (Do not allocate media file (up to 8MB) as string into memory) - Use #bytesize instead of #size to calculate file size for JSON (Fix FileOverflow error) - Segment media into subfolders by status ID because apparently GIF-to-MP4 media are all named "media.mp4" for some reason * Keep uniquely generated filename in Paperclip::GifTranscoder * Ensure dumped files do not overwrite each other by maintaing directory partitions * Give tar archives a good name * Add scheduler to remove week-old backups * Fix code style issue	7 years ago
Akihiko Odaki	613e7c7521	Rename ResolveRemoteAccountService to ResolveAccountService (#6327 ) The service used to be named ResolveRemoteAccountService resolves local accounts as well.	7 years ago
ThibG	a0de3222dd	Retry delivering toots over ActivityPub for about 2 days (#6298 ) Currently, Mastodon will retry delivering toots for a bit over 1 hour. This is a very short timespan when considering private and direct toots, which cannot be seen by the recipient at all after the delivery attempts have failed. Ideally, private and direct toots should have a different number of retries, but I do not know how to do that.	7 years ago
Eugen Rochko	7badad7797	Fix home regeneration (#6251 ) * Fix regeneration marker not being removed after completion * Return HTTP 206 from /api/v1/timelines/home if regeneration in progress Prioritize RegenerationWorker by putting it into default queue * Display loading indicator and poll home timeline while it regenerates * Add graphic to regeneration message * Make "not found" indicator consistent with home regeneration	7 years ago
Eugen Rochko	ed867eca9d	Move e-mail digest task to sidekiq, reduce workload, improve hint (#6252 )	7 years ago
ThibG	10f6793fd0	Fix PuSH workers (#6200 )	7 years ago
Eugen Rochko	d68868ca14	Lists redis clean-up (#5886 ) * When list is deleted, remove feed from redis * Clean up list feeds of inactive users	7 years ago
Eugen Rochko	85e97ecab6	Fix too many forwards (#5854 ) * Avoid sending explicit Undo->Announce when original deleted * Do not forward a reply back to the server that sent it * Deduplicate inboxes of rebloggers' followers for delete forwarding * Adjust test * Fix wrong class, bad SQL, wrong variable, outdated comment	7 years ago
Eugen Rochko	24cafd73a2	Lists (#5703 ) * Add structure for lists * Add list timeline streaming API * Add list APIs, bind list-account relation to follow relation * Add API for adding/removing accounts from lists * Add pagination to lists API * Add pagination to list accounts API * Adjust scopes for new APIs - Creating and modifying lists merely requires "write" scope - Fetching information about lists merely requires "read" scope * Add test for wrong user context on list timeline * Clean up tests	7 years ago
ThibG	2b1190065c	Retry thread resolving (#5599 ) Thread resolving is one of the few tasks that isn't retried on failure. One common cause for failure of this task is a well-connected user replying to a toot from a little-connected user on a small instance: the small instance will get many requests at once, and will often fail to answer requests within the 10 seconds timeout used by Mastodon. This changes makes the ThreadResolveWorker retry a few times, with a rapidly-increasing time before retries and large random contribution in order to spread the load over time.	7 years ago
Eugen Rochko	1032f3994f	Add ability to disable login and mark accounts as memorial (#5615 ) Fix #5597	7 years ago
aschmitz	554c2fd8af	Clean up reblog tracking keys, related improvements (#5428 ) * Clean up reblog-tracking sets from FeedManager Builds on #5419, with a few minor optimizations and cleanup of sets after they are no longer needed. * Update tests, fix multiply-reblogged case Previously, we would have lost the fact that a given status was reblogged if the displayed reblog of it was removed, now we don't. Also added tests to make sure FeedManager#trim cleans up our reblog tracking keys, fixed up FeedCleanupScheduler to use the right loop, and fixed the test for it.	7 years ago
Eugen Rochko	34118169ac	Keep references to all reblogs of a status on home feed (#5419 ) * Keep references to all reblogs of a status on home feed When inserting reblog: Add to set of reblogs of this status on the feed, if original status was present in the feed, add it to that set as well. When removing a reblog: Remove it from that set. Take random remaining item from the set. If one exists, re-insert it into feed, otherwise do not re-insert anything. Fix #4210 * When original is removed, toss out reblog references	7 years ago
abcang	2eab41cd1a	Close connection when succeeded posting (#5390 ) * Close connection when succeeded posting * Update webmock	7 years ago
Eugen Rochko	0717d9b3e6	Set snowflake IDs for backdated statuses (#5260 ) - Rename Mastodon::TimestampIds into Mastodon::Snowflake for clarity - Skip for statuses coming from inbox, aka delivered in real-time - Skip for statuses that claim to be from the future	7 years ago
unarist	5c8ca024ef	Improve error handling on LinkCrawlWorker (#5250 ) * Improve error handling on LinkCrawlWorker * Ignore TimeoutError and InvalidURIError too * Record errors to debug log * Enable dead job queue on LinkCrawlWorker Since most of acceptable errors were already ignored, only our side issue should go to dead job queue. * Ignore all http gem errors	7 years ago
Eugen Rochko	cdd5ef691b	Use separate workers to process imports, retry failures (#5207 )	7 years ago
ThibG	f7c909e290	Retry ActivityPub delivery a few more times (#5014 )	7 years ago
Eugen Rochko	f4ca116ea8	After 7 days of repeated delivery failures, give up on inbox (#5131 ) - A successful delivery cancels it out - An incoming delivery from account of the inbox cancels it out	7 years ago
abcang	3d9b8847d2	Flush body when POST requests (#5128 )	7 years ago
Eugen Rochko	e528114c53	Follow-up to #4582 and #5027 , removing dead code (#5101 )	7 years ago
Daigo 3 Dango	a0bbeafb04	Suppress backtrace when failed to communicate with a remote instance (#5076 )	7 years ago
unarist	0698c610a6	Fix an error in ReplyDistributionWorker when replied status was deleted (#4974 ) Reply distribution is proceed by Sidekiq, so replied status may be deleted before this.	8 years ago
sdukhovni	67559361e8	Add scheduled worker to purge old user IPs (#4951 ) * Add scheduled worker to purge old user IPs * Use ruby 1.9 hash syntax	8 years ago
abcang	1aad015bbb	Revert unique retry job (#4937 ) * Revert "Enable UniqueRetryJobMiddleware even when called from sidekiq worker (#4836)" This reverts commit `6859d4c028`. * Revert "Do not execute the job with the same arguments as the retry job (#4814)" This reverts commit `be7ffa2d75`.	8 years ago
ThibG	af00220d79	Fix refollowing (#4931 ) * Make RefollowWorker ActivityPub-only to avoid potential identifier mismatches * Don't call RefollowWorker on new accounts	8 years ago
ThibG	f29918e707	[WiP] Whenever a remote keypair changes, unfollow them and re-subscribe to … (#4907 ) * Whenever a remote keypair changes, unfollow them and re-subscribe to them In Mastodon (it could be different for other OStatus or AP-enabled software), a keypair change is indicative of whole user (or instance) data loss. In this situation, the “new” user might be different, and almost certainly has an empty followers list. In this case, Mastodon instances will disagree on follower lists, leading to unreliable delivery and “shadow followers”, that is users believed by a remote instance to be followers, without the affected user knowing. Drawbacks of this change are: 1. If an user legitimately changes public key for some reason without losing data (not possible in Mastodon at the moment), they will have their remote followers unsubscribed/re-subscribed needlessly. 2. Depending of the number of remote followers, this may generate quite some traffic. 3. If the user change is an attempt at usurpation, the remote followers will unknowingly follow the usurper. Note that this is not a change of behavior, Mastodon already behaves like that, although delivery might be unreliable, and the usurper would not have known the former user's followers. * Rename ResubscribeWorker to RefollowWorker * Process followers in batches	8 years ago
Eugen Rochko	7d7844a47f	Default follows for new users (#4871 ) When a new user confirms their e-mail, bootstrap their home timeline by automatically following a set of accounts. By default, all local admin accounts (that are unlocked). Can be customized by new admin setting (comma-separated usernames, local and unlocked only)	8 years ago
Eugen Rochko	e7adbf572a	Switch to static URIs, new URI format in both protocols for new statuses (#4815 ) * Decouple Status#local? from uri being nil * Replace on-the-fly URI generation with stored URIs - Generate URI in after_save hook for local statuses - Use static value in TagManager when available, fallback to tag format - Make TagManager use ActivityPub::TagManager to understand new format - Adjust tests * Use other heuristic for locality of old statuses, do not perform long query * Exclude tombstone stream entries from Atom feed * Prevent nil statuses from landing in Pubsubhubbub::DistributionWorker * Fix URI not being saved (#4818) * Add more specs for Status * Save generated uri immediately and also fix method order to minimize diff. * Fix alternate HTML URL in Atom * Fix tests * Remove not-null constraint from statuses migration to speed it up	8 years ago
abcang	be7ffa2d75	Do not execute the job with the same arguments as the retry job (#4814 )	8 years ago
Eugen Rochko	a187dcefa1	Instantly upgrade account to ActivityPub if we receive ActivityPub payload (#4766 )	8 years ago
Eugen Rochko	7b8f262840	Forward ActivityPub creates that reply to local statuses (#4709 ) * Forward ActivityPub creates that reply to local statuses * Fix test * Fix wrong signers	8 years ago
Eugen Rochko	4c76402ba1	Serialize ActivityPub alternate link into OStatus deletes, handle it (#4730 ) Requires moving Atom rendering from DistributionWorker (where `stream_entry.status` is already nil) to inline (where `stream_entry.status.destroyed?` is true) and distributing that. Unfortunately, such XML renderings can no longer be easily chained together into one payload of n items.	8 years ago
Eugen Rochko	0397c58b61	Forward ActivityPub deletes to followers of rebloggers (#4706 )	8 years ago
Eugen Rochko	00840f4f2e	Add handling of Linked Data Signatures in payloads (#4687 ) * Add handling of Linked Data Signatures in payloads * Add a way to sign JSON, fix canonicalization of signature options * Fix signatureValue encoding, send out signed JSON when distributing * Add missing security context	8 years ago
masarakki	649a20ab46	authorize-follow-requests-after-unlocking (#4658 )	8 years ago
Eugen Rochko	cf615abbf9	Add configuration to disable private status federation over PuSH (#4582 )	8 years ago

1 2 3 4

163 Commits (9a794067f77d936783736574640b1238bb8e6b18)