301 Commits (2d1f082bb6bee89242ee8042dc19016179078566)

Author	SHA1	Message	Date
Takeshi Umeda	706972b4ba	Fix RedownloadMediaWorker (#13741)	5 years ago
Eugen Rochko	e9ecbca70d	Fix error within error when limiting backtrace to 3 lines (#13120) ... Fix #13086, close #13113	5 years ago
Takeshi Umeda	04c8d825f6	Fix DeliveryWorker not to call failure_tracker when inbox_url is unavailable (#13482)	5 years ago
Eugen Rochko	5edff32733	Change delivery failure tracking to work with hostnames instead of URLs (#13437)	5 years ago
Eugen Rochko	0773a512ac	Fix trying to delete already deleted file when post-processing (#13406) ... Fix #13403	5 years ago
ThibG	89e28c7674	Fix PostgreSQL load when linking in announcements (#13250) ... * Fix PostgreSQL load when linking in announcements Fixes #13245 by caching status lookups Since statuses are supposed to be known already and we only need their URLs and a few other things, caching them should be fine. Since it's only used by announcements so far, there won't be much statuses to cache. * Perform status lookup when saving announcements, not when rendering them * Change EntityCache#status to fetch URLs instead of looking into the database * Move announcement link lookup to publishing worker * Address issues pointed out during review	5 years ago
Eugen Rochko	9014367bd8	Fix background jobs not using locks like they are supposed to (#13361) ... Also: - Fix locks not being removed when jobs go to the dead job queue - Add UI for managing locks to the Sidekiq dashboard - Remove unused Sidekiq workers Fix #13349	5 years ago
Eugen Rochko	9660aa4543	Change local media attachments to perform heavy processing asynchronously (#13210) ... Fix #9106	5 years ago
Eugen Rochko	663ea84b08	Add publish/unpublish controls to announcements in admin UI (#12967)	5 years ago
Eugen Rochko	401f32f9ee	Fix expired announcements being re-published (#12964)	5 years ago
Eugen Rochko	b9d74d4076	Add streaming API updates for announcements being modified or deleted (#12963) ... Change `all_day` to be a visual client-side cue only Publish immediately if `scheduled_at` is in the past Add `published_at` and `updated_at` to announcements JSON	5 years ago
ThibG	288f509424	Fix type error in announcement reaction update event payload (#12946)	5 years ago
Eugen Rochko	f52c988e12	Add announcements (#12662) ... * Add announcements Fix #11006 * Add reactions to announcements * Add admin UI for announcements * Add unit tests * Fix issues - Add `with_dismissed` param to announcements API - Fix end date not being formatted when time range is given - Fix announcement delete causing reactions to send streaming updates - Fix announcements container growing too wide and mascot too small - Fix `all_day` being settable when no time range is given - Change text "Update" to "Announcement" * Fix scheduler unpublishing announcements before they are due * Fix filter params not being passed to announcements filter	5 years ago
Daigo 3 Dango	e9ea09d173	Suppress backtrace when delivering toots (#12798) ... This is to suppress irrelevant backtrace from errors raised when delivering toots to remote servers. The errors are usually out of control by the local server and backtraces don't provide much information. This is similar to https://github.com/tootsuite/mastodon/pull/5174 and shortens backtraces like below: ``` WARN: Mastodon::UnexpectedResponseError: https://example.com/inbox returned code 523 WARN: app/workers/activitypub/delivery_worker.rb:48:in `block (3 levels) in perform_request' app/lib/request.rb:75:in `perform' app/workers/activitypub/delivery_worker.rb:47:in `block (2 levels) in perform_request' app/lib/request_pool.rb:53:in `use' app/lib/request_pool.rb:108:in `block (2 levels) in with' vendor/bundle/ruby/2.7.0/gems/activesupport-5.2.4.1/lib/active_support/notifications.rb:170:in `instrument' app/lib/request_pool.rb:107:in `block in with' app/lib/connection_pool/shared_connection_pool.rb:21:in `block (2 levels) in with' app/lib/connection_pool/shared_connection_pool.rb:20:in `handle_interrupt' app/lib/connection_pool/shared_connection_pool.rb:20:in `block in with' app/lib/connection_pool/shared_connection_pool.rb:16:in `handle_interrupt' app/lib/connection_pool/shared_connection_pool.rb:16:in `with' app/lib/request_pool.rb:106:in `with' app/workers/activitypub/delivery_worker.rb:46:in `block in perform_request' vendor/bundle/ruby/2.7.0/gems/stoplight-2.2.0/lib/stoplight/light/runnable.rb:51:in `run_code' vendor/bundle/ruby/2.7.0/gems/stoplight-2.2.0/lib/stoplight/light/runnable.rb:42:in `run_yellow' vendor/bundle/ruby/2.7.0/gems/stoplight-2.2.0/lib/stoplight/light/runnable.rb:24:in `run' app/workers/activitypub/delivery_worker.rb:57:in `perform_request' app/workers/activitypub/delivery_worker.rb:25:in `perform' vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:192:in `execute_job' vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:165:in `block (2 levels) in process' vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/middleware/chain.rb:128:in `block in invoke' vendor/bundle/ruby/2.7.0/gems/nsa-0.2.7/lib/nsa/collectors/sidekiq.rb:31:in `block in call' vendor/bundle/ruby/2.7.0/gems/nsa-0.2.7/lib/nsa/statsd/publisher.rb:27:in `statsd_time' vendor/bundle/ruby/2.7.0/gems/nsa-0.2.7/lib/nsa/collectors/sidekiq.rb:30:in `call' vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/middleware/chain.rb:130:in `block in invoke' app/lib/sidekiq_error_handler.rb:5:in `call' vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/middleware/chain.rb:130:in `block in invoke' vendor/bundle/ruby/2.7.0/gems/scout_apm-2.3.0.pre3/lib/scout_apm/background_job_integrations/sidekiq.rb:69:in `call' vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/middleware/chain.rb:130:in `block in invoke' vendor/bundle/ruby/2.7.0/gems/sidekiq-unique-jobs-6.0.18/lib/sidekiq_unique_jobs/server/middleware.rb:29:in `call' vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/middleware/chain.rb:130:in `block in invoke' vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/middleware/chain.rb:133:in `invoke' vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:164:in `block in process' vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:137:in `block (6 levels) in dispatch' vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/job_retry.rb:109:in `local' vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:136:in `block (5 levels) in dispatch' vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/rails.rb:43:in `block in call' vendor/bundle/ruby/2.7.0/gems/activesupport-5.2.4.1/lib/active_support/execution_wrapper.rb:87:in `wrap' vendor/bundle/ruby/2.7.0/gems/activesupport-5.2.4.1/lib/active_support/reloader.rb:73:in `block in wrap' vendor/bundle/ruby/2.7.0/gems/activesupport-5.2.4.1/lib/active_support/execution_wrapper.rb:87:in `wrap' vendor/bundle/ruby/2.7.0/gems/activesupport-5.2.4.1/lib/active_support/reloader.rb:72:in `wrap' vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/rails.rb:42:in `call' vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:132:in `block (4 levels) in dispatch' vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:250:in `stats' vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:127:in `block (3 levels) in dispatch' vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/job_logger.rb:8:in `call' vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:126:in `block (2 levels) in dispatch' vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/job_retry.rb:74:in `global' vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:125:in `block in dispatch' vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/logging.rb:48:in `with_context' vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/logging.rb:42:in `with_job_hash_context' vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:124:in `dispatch' vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:163:in `process' vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:83:in `process_one' vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:71:in `run' vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/util.rb:16:in `watchdog' vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/util.rb:25:in `block in safe_thread' ``` ``` WARN: Stoplight::Error::RedLight: https://example.com/inbox WARN: vendor/bundle/ruby/2.7.0/gems/stoplight-2.2.0/lib/stoplight/light/runnable.rb:46:in `run_red' vendor/bundle/ruby/2.7.0/gems/stoplight-2.2.0/lib/stoplight/light/runnable.rb:25:in `run' app/workers/activitypub/delivery_worker.rb:57:in `perform_request' app/workers/activitypub/delivery_worker.rb:25:in `perform' vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:192:in `execute_job' vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:165:in `block (2 levels) in process' vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/middleware/chain.rb:128:in `block in invoke' vendor/bundle/ruby/2.7.0/gems/nsa-0.2.7/lib/nsa/collectors/sidekiq.rb:31:in `block in call' vendor/bundle/ruby/2.7.0/gems/nsa-0.2.7/lib/nsa/statsd/publisher.rb:27:in `statsd_time' vendor/bundle/ruby/2.7.0/gems/nsa-0.2.7/lib/nsa/collectors/sidekiq.rb:30:in `call' vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/middleware/chain.rb:130:in `block in invoke' app/lib/sidekiq_error_handler.rb:5:in `call' vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/middleware/chain.rb:130:in `block in invoke' vendor/bundle/ruby/2.7.0/gems/scout_apm-2.3.0.pre3/lib/scout_apm/background_job_integrations/sidekiq.rb:69:in `call' vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/middleware/chain.rb:130:in `block in invoke' vendor/bundle/ruby/2.7.0/gems/sidekiq-unique-jobs-6.0.18/lib/sidekiq_unique_jobs/server/middleware.rb:29:in `call' vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/middleware/chain.rb:130:in `block in invoke' vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/middleware/chain.rb:133:in `invoke' vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:164:in `block in process' vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:137:in `block (6 levels) in dispatch' vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/job_retry.rb:109:in `local' vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:136:in `block (5 levels) in dispatch' vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/rails.rb:43:in `block in call' vendor/bundle/ruby/2.7.0/gems/activesupport-5.2.4.1/lib/active_support/execution_wrapper.rb:87:in `wrap' vendor/bundle/ruby/2.7.0/gems/activesupport-5.2.4.1/lib/active_support/reloader.rb:73:in `block in wrap' vendor/bundle/ruby/2.7.0/gems/activesupport-5.2.4.1/lib/active_support/execution_wrapper.rb:87:in `wrap' vendor/bundle/ruby/2.7.0/gems/activesupport-5.2.4.1/lib/active_support/reloader.rb:72:in `wrap' vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/rails.rb:42:in `call' vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:132:in `block (4 levels) in dispatch' vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:250:in `stats' vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:127:in `block (3 levels) in dispatch' vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/job_logger.rb:8:in `call' vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:126:in `block (2 levels) in dispatch' vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/job_retry.rb:74:in `global' vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:125:in `block in dispatch' vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/logging.rb:48:in `with_context' vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/logging.rb:42:in `with_job_hash_context' vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:124:in `dispatch' vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:163:in `process' vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:83:in `process_one' vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:71:in `run' vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/util.rb:16:in `watchdog' vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/util.rb:25:in `block in safe_thread' ```	5 years ago
ThibG	aa138ea350	Fix RefollowWorker not keeping show_reblogs setting (#12707) ... * Fix RefollowWorker not keeping show_reblogs setting * Fix RefollowWorker	5 years ago
ThibG	66c1fe0495	Fix various issues with account migration (#12301) ... * Fix being able to follow oneself by moving to an account that was following the old one * Add specs * Add spec to catch MoveWorker issue with local followers following both accounts * Fix move worker breaking when a local account follows both source and target accounts * Fix migration from remote to local account not sending Undo Follow * Fix show_reblogs not being preserved for moved account's followers	5 years ago
Eugen Rochko	9a1be333b8	Fix wrong attribute in MoveWorker (#12066)	5 years ago
Eugen Rochko	368a87755b	Fix account migration not affecting followers on origin server (#11980)	6 years ago
Takeshi Umeda	a90243a712	Fixed missing account in MoveDistributionWorker (#11906)	6 years ago
Eugen Rochko	3ed94dcc1a	Add account migration UI (#11846) ... Fix #10736 - Change data export to be available for non-functional accounts - Change non-functional accounts to include redirecting accounts	6 years ago
Eugen Rochko	38dc51b2d6	Fix Move handler queuing jobs that will fail if account is suspended (#11864) ... Don't put Move handler on cooldown if it didn't run. Skip unmerging from timelines to save unnecessary work.	6 years ago
abcang	c21386cff5	Improve IP cleanup query (#11871)	6 years ago
Eugen Rochko	e8703c281e	Fix web push subscriptions being deleted on rate limit or timeout (#11826)	6 years ago
Eugen Rochko	c5d37f18cb	Change deletes to preserve soft-deleted statuses in unresolved reports (#11805) ... Change all account actions except "none" to resolve all unresolved reports Refactor `SuspendAccountService` to be more readable	6 years ago
Eugen Rochko	031ca25014	Add retry for failed media downloads and `tootctl media refresh` (#11775)	6 years ago
Eugen Rochko	8674814825	Change tootctl to use inline parallelization instead of Sidekiq (#11776) ... - Remove --background option - Add --concurrency(=5) option - Add progress bars	6 years ago
Eugen Rochko	70ddef2654	Change trending hashtags to not disappear instantly after midnight (#11712)	6 years ago
Eugen Rochko	282ea17078	Add soft delete for statuses for instant deletes through API (#11623) ... * Add soft delete for statuses to allow them to appear instant * Allow reporting soft-deleted statuses and show them in the admin UI * Change index for getting an account's statuses	6 years ago
Eugen Rochko	97192d9a77	Fix remote and staff-removed statuses leaving media behind for a day (#11638) ... The reason for unattaching media instead of removing it is to support delete & redraft functionality, but remote or staff-removed statuses will never be redrafted, so the media should be deleted immediately	6 years ago
ThibG	bced70469a	Add domain block notes (#11515) ... * Add database columns for adding notes to domain blocks/restrctions * Add admin UI to set private and public comments when blocking a domain * Add text for private and public comments on domain blocks * Show domain block comments in admin UI * Add comments to the domain block undo page * Make UnblockDomainService more robust regarding upgraded domain blocks * Allow editing domain blocks * Rename button from “undo domain block” to “view domain block” in account admin UI * Change test to unsilence silenced users from upgraded blocks	6 years ago
mayaeh	0d80f686d8	Add `tootctl preview_cards remove` (#11320) ... * Add `tootctl preview_cards remove` * fix code style * Remove `Scheduler::PreviewCardsCleanupScheduler` file * fix code style again Add exclude case where image_file_name is blank * Added a function to output confirmation if the specified number of days is less than 2 weeks	6 years ago
Eugen Rochko	bd87e66679	Remove WebSub subscriptions (#11303)	6 years ago
Eugen Rochko	b3f44aa186	Add periodic removal of older thumbnails for preview cards (#11304)	6 years ago
han@highemelry	402302776c	Change the retry limit in error of web push notification (#11292) ... - Change the maximum count of retry for web push notification (Default -> 5). - In case of high load of subscribe server, the retries will be repeated many times. - Because the retries occupy the default queue, maximum retry count should be reduced.	6 years ago
Eugen Rochko	5d3feed191	Refactor fetching of remote resources (#11251)	6 years ago
Eugen Rochko	406b46395d	Fix URLs appearing twice in errors of ActivityPub::DeliveryWorker (#11231)	6 years ago
Eugen Rochko	23aeef52cc	Remove Salmon and PubSubHubbub (#11205) ... * Remove Salmon and PubSubHubbub endpoints * Add error when trying to follow OStatus accounts * Fix new accounts not being created in ResolveAccountService	6 years ago
Eugen Rochko	bc60d794f8	Change ActivityPub::DeliveryWorker to not retry HTTP 501 errors (#11233)	6 years ago
Eugen Rochko	0d9ffe56fb	Add request pool to improve delivery performance (#10353) ... * Add request pool to improve delivery performance Fix #7909 * Ensure connection is closed when exception interrupts execution * Remove Timeout#timeout from socket connection * Fix infinite retrial loop on HTTP::ConnectionError * Close sockets on failure, reduce idle time to 90 seconds * Add MAX_REQUEST_POOL_SIZE option to limit concurrent connections to the same server * Use a shared pool size, 512 by default, to stay below open file limit * Add some tests * Add more tests * Reduce MAX_IDLE_TIME from 90 to 30 seconds, reap every 30 seconds * Use a shared pool that returns preferred connection but re-purposes other ones when needed * Fix wrong connection being returned on subsequent calls within the same thread * Reduce mutex calls on flushes from 2 to 1 and add test for reaping	6 years ago
Eugen Rochko	f2b743e715	Refactor all ActivityPub deliveries to be serialized and signed through one concern (#10966)	6 years ago
ThibG	9efcca3c54	Retry ActivityPub inbox delivery on HTTP 401 and 408 errors (#10812) ... HTTP 401 responses returned by Mastodon's inbox controller may be temporary if, for instance, the requesting user's actor/key json could not be retrieved in a timely fashion. This changes allow retries instead of dropping the message entirely. Also added HTTP 408 as that error is by nature temporary.	6 years ago
ThibG	b85f216cbc	Do not retry processing ActivityPub jobs raising validation errors (#10614) ... * Do not retry processing ActivityPub jobs raising validation errors Jobs yielding validation errors most probably won't ever be accepted, so it makes sense not to clutter the queues with retries. * Lower RecordInvalid error reporting to debug log level * Remove trailing whitespace	6 years ago
ThibG	cb71c95e22	Export and import show_reblogs together with following list (#10495) ... * Refactor imports * Export show_reblogs when exporting list of followed users * Add support for importing show_reblogs with following collection * Fix tests	6 years ago
ThibG	d4882aa64a	Export and import `hide_notifications` alongside user mutes (#10335) ... * Export hide_notifications along with user mutes * Import hide_notifications along with muted users list * Add headers for CSV exports	6 years ago
Eugen Rochko	f1bc90ab50	Rename :poll to :preloadable_poll and :owned_poll to :poll on Status (#10401) ... Also, fix some n+1 queries Resolve #10365	6 years ago
ThibG	6b0eda14a1	Avoid race condition when streaming deleted statuses (#10280) ... * Avoid race condition when streaming deleted statuses * Move redis lock to DistributionWorker to avoid extra Redis value	6 years ago
Eugen Rochko	6841d8fc74	Fix wrong method used in PollExpirationNotifyWorker (#10265)	6 years ago
Eugen Rochko	9f5b55ad4f	Fix poll update handler calling method was that was not available (#10246) ... * Fix poll update handler calling method was that was not available Fix regression from #10209 * Refactor VoteService * Refactor ActivityPub::DistributePollUpdateWorker and optimize it * Fix typo * Fix typo	6 years ago
ThibG	3a92885a86	Support pushing and receiving updates to poll tallies (#10209) ... * Process incoming poll tallies update * Send Update on poll vote * Do not send Updates for a poll more often than once every 3 minutes * Include voters in people to notify of results update * Schedule closing poll worker on poll creation * Add new notification type for ending polls * Add front-end support for ended poll notifications * Fix UpdatePollSerializer * Fix Updates not being triggered by local votes * Fix tests failure * Fix web push notifications for closing polls * Minor cleanup * Notify voters of both remote and local polls when those close * Fix delivery of poll updates to mentioned accounts and voters	6 years ago
ThibG	df5924a1db	Do not error out on unsalvageable errors in FetchRepliesService (#10175) ... * Do not error out on unsalvageable errors in FetchRepliesService Fixes #10152 * Fix FetchRepliesWorker erroring out on deleted statuses	6 years ago
ThibG	9d3c6f1849	Improved remote thread fetching (#10106) ... * Fetch up to 5 replies when discovering a new remote status This is used for resolving threads downwards. The originating server must add a “replies” attributes with such replies for it to be useful. * Add some tests for ActivityPub::FetchRepliesWorker * Add specs for ActivityPub::FetchRepliesService * Serialize up to 5 public self-replies for ActivityPub notes * Add specs for ActivityPub::NoteSerializer * Move exponential backoff logic to a worker concern * Fetch first page of paginated collections when fetching thread replies * Add specs for paginated collections in replies * Move Note replies serialization to a first CollectionPage The collection isn't actually paginable yet as it has no id nor a `next` field. This may come in another PR. * Use pluck(:uri) instead of map(&:uri) to improve performances * Fix fetching replies when they are in a CollectionPage	6 years ago
Eugen Rochko	dad339da6d	Filter incoming Create activities by relation to local activity (#10005) ... Reject those from accounts with no local followers, from relays that are not enabled, which do not address local accounts and are not replies to accounts that do have local followers	6 years ago
ThibG	99c4b5b75a	Move sending account Delete to anyone but the account's followers to the pull̀ queue (#10016)	6 years ago
Eugen Rochko	d14c276e58	Add option to overwrite imported data (#9962) ... * Add option to overwrite imported data Fix #7465 * Add import for domain blocks	6 years ago
ysksn	bcfff65195	Create Redisable#redis (#9633) ... * Create Redisable * Use #redis instead of Redis.current	6 years ago
Eugen Rochko	31e7940de5	Fix multiple workers for the same scheduled status being queueable (#9875)	6 years ago
Eugen Rochko	1cbdf8d218	Fix wrong param name in scheduled statuses and return params in API (#9725) ... The database column and API param are called in_reply_to_id, not in_reply_to_status_id, so it makes no sense to encode it that way	6 years ago
Eugen Rochko	8ad57a0426	Fix PublishScheduledStatusWorker being queued with no arguments (#9724)	6 years ago
Eugen Rochko	a49d43d112	Add scheduled statuses (#9706) ... Fix #340	6 years ago
Eugen Rochko	b17b2f25ac	Follow new account before unfollowing the old one during move (#9719)	6 years ago
Eugen Rochko	66436d0895	Improve e-mail digest (#9689) ... - Reduce time-to-digest from 20 to 7 days - Fetch mentions starting from +1 day since last login - Fix case when last login is more recent than last e-mail - Do not render all mentions, only 40, but show number in subject - Do not send digest to moved accounts - Do send digest to silenced accounts	6 years ago
ThibG	1d4215be77	Restore ReplyDistributionWorker to allow existing jobs to be processed (#9676)	6 years ago
ThibG	6fb6a53938	Avoid duplicate work by merging ReplyDistributionWorker into DistributionWorker (#9660)	6 years ago
ThibG	290932602b	Reduce usage of LD signatures (#9659) ... * Do not LDS-sign Follow, Accept, Reject, Undo, Block * Do not use LDS for Create activities of private toots * Minor cleanup * Ignore unsigned activities instead of misattributing them * Use status.distributable? instead of querying visibility directly	6 years ago
Eugen Rochko	0f938ff29c	Add handler for Move activity (#9629)	6 years ago
Eugen Rochko	628da11e38	Do no retry web push workers if the server returns a 4xx response (#9434) ... Add timeout of 10s to web push requests	6 years ago
Eugen Rochko	6ddf0432e7	Improve account suspension speed and completeness (#9290) ... - Some associations were missing from the clean-up - Some attributes were not reset on suspension - Skip federation and streaming deletes when purging a dead domain - Move account association definitions to concern	6 years ago
Eugen Rochko	11955600ad	Skip deliveries to inboxes that have already been marked as unavailable (#9358)	6 years ago
Eugen Rochko	6d59dfa15d	Optimize the process of following someone (#9220) ... * Eliminate extra accounts select query from FollowService * Optimistically update follow state in web UI and hide loading bar Fix #6205 * Asynchronize NotifyService in FollowService And fix failing test * Skip Webfinger resolve routine when called from FollowService if possible If an account is ActivityPub, then webfinger re-resolving is not necessary when called from FollowService. Improve options of ResolveAccountService	6 years ago
Eugen Rochko	d4cf963749	Allow inbox owner to view implicitly targeted ActivityPub payload (#9093) ... Fix #9091	6 years ago
Eugen Rochko	ddd30f331c	Improve support for aspects/circles (#8950) ... * Add silent column to mentions * Save silent mentions in ActivityPub Create handler and optimize it Move networking calls out of the database transaction * Add "limited" visibility level masked as "private" in the API Unlike DMs, limited statuses are pushed into home feeds. The access control rules between direct and limited statuses is almost the same, except for counter and conversation logic * Ensure silent column is non-null, add spec * Ensure filters don't check silent mentions for blocks/mutes As those are "this person is also allowed to see" rather than "this person is involved", therefore does not warrant filtering * Clean up code * Use Status#active_mentions to limit returned mentions * Fix code style issues * Use Status#active_mentions in Notification And remove stream_entry eager-loading from Notification	6 years ago
Eugen Rochko	7f9dd92a27	Track historical space stats in PgHero to determine PostgreSQL growth (#8906)	6 years ago
Eugen Rochko	774ac47373	Add conversations API (#8832) ... * Add conversations API * Add web UI for conversations * Add test for conversations API * Add tests for ConversationAccount * Improve web UI * Rename ConversationAccount to AccountConversation * Remove conversations on block and mute * Change last_status_id to be a denormalization of status_ids * Add optimistic locking	6 years ago
Eugen Rochko	a46ab86adf	Limit the number of people that can be followed from one account (#8807) ... Configurable soft limit of 7,500, and above that, configurable ratio of 1.1 * followers, controlled by: - MAX_FOLLOWS_THRESHOLD - MAX_FOLLOWS_RATIO Fix #2311	7 years ago
Eugen Rochko	f4d549d300	Redesign forms, verify link ownership with rel="me" (#8703) ... * Verify link ownership with rel="me" * Add explanation about verification to UI * Perform link verifications * Add click-to-copy widget for verification HTML * Redesign edit profile page * Redesign forms * Improve responsive design of settings pages * Restore landing page sign-up form * Fix typo * Support <link> tags, add spec * Fix links not being verified on first discovery and passive updates	7 years ago
Eugen Rochko	cabdbb7f9c	Add CLI task for rotating keys (#8466) ... * If an Update is signed with known key, skip re-following procedure Because it means the remote actor did *not* lose their database * Add CLI method for rotating keys bin/tootctl accounts rotate [USERNAME] Generates a new RSA key per account and sends out an Update activity signed with the old key. * Key rotation: Space out Update fan-outs every 5 minutes per 1000 accounts * Skip suspended accounts in key rotation	7 years ago
Eugen Rochko	93537c5560	Disable retries on all scheduled tasks (#8413)	7 years ago
Eugen Rochko	793eea2982	Add improved CLI interface for removing remote media (#8411) ... ./bin/tootctl media remove --days 7 --background Make the old rake task point to it	7 years ago
ThibG	f06fa09962	Revert to using Paperclip's filesystem storage, and fix dangling records in remove_remote (#8339) ... * Fix uncaching worker * Revert to using Paperclip's filesystem backend instead of fog-local fog-local has lots of concurrency issues, causing failure to delete files, dangling file records, and spurious errors UncacheMediaWorker	7 years ago
Eugen Rochko	d98de8ada7	Get rid of all batch order warnings (#8334)	7 years ago
Eugen Rochko	91c929a42c	Keep scheduler jobs unique until they're done (#8287)	7 years ago
Eugen Rochko	7e2678f7f6	Fix ActivityPub::UpdateDistributionWorker regression (#8039) ... Regression from #7998 let to profile updates not sending	7 years ago
Eugen Rochko	e55dce3176	Add federation relay support (#7998) ... * Add federation relay support * Add admin UI for managing relays * Include actor on relay-related activities * Fix i18n	7 years ago
Eugen Rochko	10f51c9886	Fix domain hiding logic (#7765) ... * Send rejections to followers when user hides domain they're on * Use account domain blocks for "authorized followers" action Replace soft-blocking (block & unblock) behaviour with follow rejection * Split sync and async work of account domain blocking Do not create domain block when removing followers by domain, that is probably unexpected from the user's perspective. * Adjust confirmation message for domain block * yarn manage:translations	7 years ago
Eugen Rochko	ad40bf5e0c	Speed up some rake tasks by moving execution to Sidekiq (#7678) ... * Speed up some rake tasks by moving execution to Sidekiq mastodon:media:remove_silenced mastodon:media:remove_remote mastodon:media:redownload_avatars mastodon:feeds:build * Fix code style issue	7 years ago
takayamaki	587da93152	checking http status code with range (#7544)	7 years ago
Eugen Rochko	97f02f2c08	Do not raise delivery failure on 4xx errors, increase stoplight threshold (#7541) ... * Do not raise delivery failure on 4xx errors, increase stoplight threshold Stoplight failure threshold from 3 to 10 Status code 429 will raise a failure/get retried * Oops	7 years ago
Eugen Rochko	b4fb766b23	Add REST API for Web Push Notifications subscriptions (#7445) ... - POST /api/v1/push/subscription - PUT /api/v1/push/subscription - DELETE /api/v1/push/subscription - New OAuth scope: "push" (required for the above methods)	7 years ago
ThibG	a24605961a	Fixes/do not override timestamps (#7336) ... * Revert "Fixes/do not override timestamps (#7331)" This reverts commit 581a5c9d29. * Document Snowflake ID corner-case a bit more Snowflake IDs are used for two purposes: making object identifiers harder to guess and ensuring they are in chronological order. For this reason, they are based on the `created_at` attribute of the object. Unfortunately, inserting items with older snowflakes IDs will break the assumption of consumers of the paging APIs that new items will always have a greater identifier than the last seen one. * Add `override_timestamps` virtual attribute to not correlate snowflake ID with created_at	7 years ago
ThibG	581a5c9d29	Fixes/do not override timestamps (#7331) ... * Do not override timestamps for incoming toots * Remove every reference to override_timestamps Statuses are now created with the announced publishing date and are only pushed to timelines if that date is at most 6 hours earlier than the time at which it is processed.	7 years ago
Eugen Rochko	d5fa4fbcd2	Revert "Do not override timestamps for incoming toots" (#7330) ... * Revert "Weblate translations 20180503 (#7325)" This reverts commit dfa6bccb64. * Revert "Prevent timeline from moving when cursor is hovering over it (fixes #7278) (#7327)" This reverts commit 58852695c8. * Revert "Add pry-byebug (#7307)" This reverts commit ab773e4d5f. * Revert "Do not override timestamps for incoming toots (#7326)" This reverts commit bd36791832.	7 years ago
ThibG	bd36791832	Do not override timestamps for incoming toots (#7326)	7 years ago
Eugen Rochko	658cbc9425	Improve PostStatusService performance (#7317) ... Offload creation of local notifications to a worker. Remove two redundant SQL queries from ProcessMentionsService, remove n+1 XML/JSON serialization via memoization	7 years ago
Eugen Rochko	cb5b5cb5f7	Slightly reduce RAM usage (#7301) ... * No need to re-require sidekiq plugins, they are required via Gemfile * Add derailed_benchmarks tool, no need to require TTY gems in Gemfile * Replace ruby-oembed with FetchOEmbedService Reduce startup by 45382 allocated objects * Remove preloaded JSON-LD in favour of caching HTTP responses Reduce boot RAM by about 6 MiB * Fix tests * Fix test suite by stubbing out JSON-LD contexts	7 years ago
Eugen Rochko	778562c223	Ensure SynchronizeFeaturedCollectionWorker is unique and clean up (#7043) ... * Ensure SynchronizeFeaturedCollectionWorker is unique and clean up Fix #7041 * Fix code style issue	7 years ago
Eugen Rochko	d4de2239b0	Add a circuit breaker for ActivityPub deliveries (#7053)	7 years ago
Eugen Rochko	f1867a7388	Adjust privacy policy (#6666) ... * Adjust privacy policy to be more specific to Mastodon Fix #6613 * Change data retention of IP addresses from 5 years to 1 year * Add even more information * Remove all (now invalid) translations of the privacy policy * Add information about archive takeout, remove pointless consent section * Emphasis on DM privacy * Improve wording * Add line about data use for moderation purposes	7 years ago
Akihiko Odaki	40e5d2303b	Validate HTTP response length while receiving (#6891) ... to_s method of HTTP::Response keeps blocking while it receives the whole content, no matter how it is big. This means it may waste time to receive unacceptably large files. It may also consume memory and disk in the process. This solves the inefficency by checking response length while receiving.	7 years ago
Akihiko Odaki	54b273bf99	Close http connection in perform method of Request class (#6889) ... HTTP connections must be explicitly closed in many cases, and letting perform method close connections makes its callers less redundant and prevent them from forgetting to close connections.	7 years ago
Eugen Rochko	9e3a6d6784	Log BackupWorker backtrace, delete Backup if retries exhausted (#6769)	7 years ago