163 Commits (9a794067f77d936783736574640b1238bb8e6b18)

Author	SHA1	Message	Date
Eugen Rochko	f391a4673a	Periodically remove expired PuSH subscribers (#4654)	8 years ago
Eugen Rochko	6e9eda5331	ActivityPub migration procedure (#4617) ... * ActivityPub migration procedure Once one account is detected as going from OStatus to ActivityPub, invalidate WebFinger cache for other accounts from the same domain * Unsubscribe from PuSH updates once we receive an ActivityPub payload * Re-subscribe to PuSH unless already unsubscribed, regardless of protocol	8 years ago
Eugen Rochko	412ea87306	Improve ActivityPub/OStatus compatibility (#4632) ... *Note: OStatus URIs are invalid for ActivityPub. But we have them for as long as we want to keep old OStatus-sourced content and as long as we remain OStatus-compatible.* - In Announce handling, if object URI is not a URL, fallback to object URL - Do not use specialized ThreadResolveWorker, rely on generalized handling - When serializing notes, if parent's URI is not a URL, use parent's URL	8 years ago
Eugen Rochko	b7370ac8ba	ActivityPub delivery (#4566) ... * Deliver ActivityPub Like * Deliver ActivityPub Undo-Like * Deliver ActivityPub Create/Announce activities * Deliver ActivityPub creates from mentions * Deliver ActivityPub Block/Undo-Block * Deliver ActivityPub Accept/Reject-Follow * Deliver ActivityPub Undo-Follow * Deliver ActivityPub Follow * Deliver ActivityPub Delete activities Incidentally fix #889 * Adjust BatchedRemoveStatusService for ActivityPub * Add tests for ActivityPub workers * Add tests for FollowService * Add tests for FavouriteService, UnfollowService and PostStatusService * Add tests for ReblogService, BlockService, UnblockService, ProcessMentionsService * Add tests for AuthorizeFollowService, RejectFollowService, RemoveStatusService * Add tests for BatchedRemoveStatusService * Deliver updates to a local account to ActivityPub followers * Minor adjustments	8 years ago
Eugen Rochko	3d47154c20	Only PuSH-resubscribe to OStatus accounts (#4583)	8 years ago
Eugen Rochko	dd7ef0dc41	Add ActivityPub inbox (#4216) ... * Add ActivityPub inbox * Handle ActivityPub deletes * Handle ActivityPub creates * Handle ActivityPub announces * Stubs for handling all activities that need to be handled * Add ActivityPub actor resolving * Handle conversation URI passing in ActivityPub * Handle content language in ActivityPub * Send accept header when fetching actor, handle JSON parse errors * Test for ActivityPub::FetchRemoteAccountService * Handle public key and icon/image when embedded/as array/as resolvable URI * Implement ActivityPub::FetchRemoteStatusService * Add stubs for more interactions * Undo activities implemented * Handle out of order activities * Hook up ActivityPub to ResolveRemoteAccountService, handle Update Account activities * Add fragment IDs to all transient activity serializers * Add tests and fixes * Add stubs for missing tests * Add more tests * Add more tests	8 years ago
Yamagishi Kazutoshi	3e7a541e09	Change RuboCop rules to loose (#4464)	8 years ago
Eugen Rochko	4e2f2fab73	Fix guard clause in WebPushNotificationWorker (#4421)	8 years ago
Eugen Rochko	6e186b9c77	When PuSH subscribe attempts are exhausted, unsubscribe (#4422)	8 years ago
Clworld	994d948c39	Add callback_url/acct information for Sidekiq PuSH workers Exception. (#4281) ... * Add destination informations to exception on SubscribeWorker and DeliveryWorker. * Simplify delivery error message. * Prevent changing Exception type... * fix typo.	8 years ago
Sorin Davidoi	8b43d6bf9c	fix(web_push_notification_worker): Guard against deleted notifications (#4379)	8 years ago
Eugen Rochko	467456f7a1	Move clean up of unconfirmed users to sidekiq-scheduler (#4336) ... * Move clean up of unconfirmed users to sidekiq-scheduler * mastodon:daily is now deprecated	8 years ago
Eugen Rochko	1fcdaafa6f	Fix webfinger retries (#4275) ... * Do not raise unretryable exceptions in ResolveRemoteAccountService * Removed fatal exceptions from ResolveRemoteAccountService Exceptions that cannot be retried should not be raised. New exception class for those that can be retried (Mastodon::UnexpectedResponseError)	8 years ago
Eugen Rochko	a5a07da892	Correct OStatus inflection (Ostatus -> OStatus) (#4255)	8 years ago
Akihiko Odaki	89b988cab5	Introduce Ostatus name space (#4164) ... * Wrap methods of ProcessFeedService::ProcessEntry in classes This is a change same with 425acecfdb, except that it has the following changes: * Revert irrelevant change in find_or_create_conversation * Fix error handling for RemoteActivity * Introduce Ostatus name space	8 years ago
Sorin Davidoi	8387b3928e	fix(push-subscriptions): Refactor how Sidekiq jobs are handled (#4226)	8 years ago
Akihiko Odaki	7f35947d8e	Improve a query in WebPushNotificationWorker (#4234)	8 years ago
Akihiko Odaki	68941d4dfa	Use update method to update session in WebPushNotificationWorker (#4235)	8 years ago
ThibG	05cd37097c	Optimize uri normalization (#4212) ... * Add dependency on idn-ruby to speed up URI normalization * Use normalized_host instead of normalize.host when applicable When we are only interested in the normalized host, calling normalized_host avoids normalizing the other components of the URI as well as creating a new object	8 years ago
Eugen Rochko	cd9b2ab2f7	Fix #2672 - Connect signed PuSH subscription requests to instance domain (#4205) ... * Fix #2672 - Connect signed PuSH subscription requests to instance domain Resolves #2739 * Fix return of locate_subscription * Fix tests	8 years ago
Eugen Rochko	1618b68bfa	HTTP signatures (#4146) ... * Add Request class with HTTP signature generator Spec: https://tools.ietf.org/html/draft-cavage-http-signatures-06 * Add HTTP signature verification concern * Add test for SignatureVerification concern * Add basic test for Request class * Make PuSH subscribe/unsubscribe requests use new Request class Accidentally fix lease_seconds not being set and sent properly, and change the new minimum subscription duration to 1 day * Make all PuSH workers use new Request class * Make Salmon sender use new Request class * Make FetchLinkService use new Request class * Make FetchAtomService use the new Request class * Make Remotable use the new Request class * Make ResolveRemoteAccountService use the new Request class * Add more tests * Allow +-30 seconds window for signed request to remain valid * Disable time window validation for signed requests, restore 7 days as PuSH subscription duration (which was previous default due to a bug)	8 years ago
Sorin Davidoi	0c7c188c45	Web Push Notifications (#3243) ... * feat: Register push subscription * feat: Notify when mentioned * feat: Boost, favourite, reply, follow, follow request * feat: Notification interaction * feat: Handle change of public key * feat: Unsubscribe if things go wrong * feat: Do not send normal notifications if push is enabled * feat: Focus client if open * refactor: Move push logic to WebPushSubscription * feat: Better title and body * feat: Localize messages * chore: Fix lint errors * feat: Settings * refactor: Lazy load * fix: Check if push settings exist * feat: Device-based preferences * refactor: Simplify logic * refactor: Pull request feedback * refactor: Pull request feedback * refactor: Create /api/web/push_subscriptions endpoint * feat: Spec PushSubscriptionController * refactor: WebPushSubscription => Web::PushSubscription * feat: Spec Web::PushSubscription * feat: Display first media attachment * feat: Support direction * fix: Stuff broken while rebasing * refactor: Integration with session activations * refactor: Cleanup * refactor: Simplify implementation * feat: Set VAPID keys via environment * chore: Comments * fix: Crash when no alerts * fix: Set VAPID keys in testing environment * fix: Follow link * feat: Notification actions * fix: Delete previous subscription * chore: Temporary logs * refactor: Move migration to a later date * fix: Fetch the correct session activation and misc bugs * refactor: Move migration to a later date * fix: Remove follow request (no notifications) * feat: Send administrator contact to push service * feat: Set time-to-live * fix: Do not show sensitive images * fix: Reducer crash in error handling * feat: Add badge * chore: Fix lint error * fix: Checkbox label overlap * fix: Check for payload support * fix: Rename action "type" (crash in latest Chrome) * feat: Action to expand notification * fix: Lint errors * fix: Unescape notification body * fix: Do not allow boosting if the status is hidden * feat: Add VAPID keys to the production sample environment * fix: Strip HTML tags from status * refactor: Better error messages * refactor: Handle browser not implementing the VAPID protocol (Samsung Internet) * fix: Error when target_status is nil * fix: Handle lack of image * fix: Delete reference to invalid subscriptions * feat: Better error handling * fix: Unescape HTML characters after tags are striped * refactor: Simpify code * fix: Modify to work with #4091 * Sort strings alphabetically * i18n: Updated Polish translation it annoys me that it's not fully localized :P * refactor: Use current_session in PushSubscriptionController * fix: Rebase mistake * fix: Set cacheName to mastodon * refactor: Pull request feedback * refactor: Remove logging statements * chore(yarn): Fix conflicts with master * chore(yarn): Copy latest from master * chore(yarn): Readd offline-plugin * refactor: Use save! and update! * refactor: Send notifications async * fix: Allow retry when push fails * fix: Save track for failed pushes * fix: Minify sw.js * fix: Remove account_id from fabricator	8 years ago
Eugen Rochko	34ccc058fa	Limit total subscribe retries to 10, but space them out more (#4142) ... Since there is little point in retrying so often when a service is down or does not exist anymore. Subscriptions are renewed 1 day before they should expire, so retrying in 30 minutes, then 2 hours, then 12 hours is fine. If even after that, the remote server does not work, there is little sense in retrying more often than once a day Also, uniqueness of the job should ensure that failed retries will not result in multiple retries for the same endpoint when the next resubscription cycle comes	8 years ago
Eugen Rochko	8b2cad5637	Refactor JSON templates to be generated with ActiveModelSerializers instead of Rabl (#4090)	8 years ago
Eugen Rochko	ed7dc1704d	Bind web UI access tokens to sessions (#3940) ... * Add overview of active sessions * Better display of browser/platform name * Improve how browser information is stored and displayed for sessions overview * Fix test * Fix #2347 - Bind web UI access token to session When you logout, session also destroys the access token, so it's no longer valid. If access token is destroyed some other way, the session is also destroyed, requiring a re-login. Fix #1681 - Add scheduler to remove revoked access tokens and grants * Fix test	8 years ago
Akihiko Odaki (@fn_aki@pawoo.net)	e27f792c24	Some minor change and spec for Account (#3813) ... * Introduce domains method to Account relation Account had followers_domains method, which was excessively specific. Let relation of Account have domains method instead. * Move follow_mapping in Account to AccountInteractions * Introduce shared examples for AccountAvatar inclusion * Cover Account more	8 years ago
Eugen Rochko	f3be605286	Rename FollowRemoteAccountService to ResolveRemoteAccountService (#3847) ... Rename Activitypub to ActivityPub	8 years ago
Eugen Rochko	e17c2e5da5	Batched remove status service (#3735) ... * Make Pubsubhubbub::DistributionWorker handle both single stream entry arguments, as well as arrays of stream entries * Add BatchedRemoveStatusService, make SuspendAccountService use it * Improve method names * Add test * Add more tests * Use PuSH payloads of 100 to have a clear mapping of 1000 input statuses -> 10 PuSH payloads It was nice while it lasted	8 years ago
Eugen Rochko	4a618908e8	Account deletion (#3728) ... * Add form for account deletion * If avatar or header are gone from source, remove them * Add option to have SuspendAccountService remove user record, add tests * Exclude suspended accounts from search	8 years ago
takayamaki	3eedad2737	change sidekiq queueing to bulk push (#3536)	8 years ago
Matt Jankowski	ec34ec63b1	Specs for cleanup workers (#3235) ... * Add spec files for feed and media cleanup workers * Add coverage for feed and media cleanup schedulers * Clean up feed and media cleanup workers	8 years ago
Eugen Rochko	ef900789bc	Replace mastodon:media:clear and mastodon:feeds:clear rake tasks with (#3180) ... sidekiq-scheduler jobs Resolves #2495	8 years ago
Matt Jankowski	d2e0edd721	Add spec coverage for regeneration worker (#3143)	8 years ago
Matt Jankowski	db4119f971	Specs for precompute feed service (#3142) ... * Add spec for precompute feed service * Refactor PrecomputeFeedService * spec wip	8 years ago
Matt Jankowski	8ec8410651	After remote follow worker specs (#3098) ... * Add specs for the after remote follow workers * Refactor the after remote follow workers	8 years ago
Eugen Rochko	657496b5a9	Do not cancel PuSH subscriptions after encountering "permanent" error… (#3046) ... * Do not cancel PuSH subscriptions after encountering "permanent" error response After talking with MMN about it, turns out some servers/php setups do return 4xx errors while rebooting, so this anti-feature that was meant to take load off of the hub is doing more harm than good in terms of breaking subscriptions * Update delivery_worker.rb	8 years ago
Matt Jankowski	0d70fe2659	Spec coverage and refactor for pubsub/delivery worker (#3021) ... * Framework for delivery worker spec * Refactor of pubsub delivery worker	8 years ago
Matt Jankowski	08949cca41	Pubsub confirmation worker spec (#2974) ... * Add specs for valid challenge response from pubsub confirmation worker * Refactor the pubsub confirmation worker	8 years ago
Matt Jankowski	cc9a6a710f	Spec feed insert worker (#2965) ... * Spec for feed insert worker when missing records * more specs! * Refactor feed insert worker	8 years ago
Matt Jankowski	d142544159	Spec coverage and refactor of digest mailer worker (#2961)	8 years ago
Eugen Rochko	3a38322a54	Add spec for Pubsubhubbub::DistributionWorker. PuSH-deliver public items (#2954) ... to all subscribers. IDN-normalize callback URLs for subscriptions on insert.	8 years ago
haoyayoi	2fba94b36e	add content-type in pubsubhubhub request header (#2943) ... * add content-type in pubsubhubhub request header * fix type	8 years ago
Eugen Rochko	b8e166894b	Fix Scheduler::SubscriptionsScheduler (#2834) ... * Fix Scheduler::SubscriptionsScheduler, add worker test for it * Change production log level of Sidekiq to "warn" instead of "info"	8 years ago
Eugen Rochko	81584779cb	More robust PuSH subscription refreshes (#2799) ... * Fix #2473 - Use sidekiq scheduler to refresh PuSH subscriptions instead of cron Fix an issue where / in domain would raise exception in TagManager#normalize_domain PuSH subscriptions refresh done in a round-robin way to avoid hammering a single server's hub in sequence. Correct handling of failures/retries through Sidekiq (see also #2613). Optimize Account#with_followers scope. Also, since subscriptions are now delegated to Sidekiq jobs, an uncaught exception will not stop the entire refreshing operation halfway through Fix #2702 - Correct user agent header on outgoing http requests * Add test for SubscribeService * Extract #expiring_accounts into method * Make mastodon:push:refresh no-op * Queues are now defined in sidekiq.yml * Queues are now in sidekiq.yml	8 years ago
Eugen	17c591ffba	Punycode URI normalization (#2370) ... * Fix #2119 - Whenever about to send a HTTP request, normalize the URI * Add test for IDN request in FetchLinkCardService * Perform IDN normalization on domains before they are stored in the DB	8 years ago
Eugen	501514960a	Followers-only post federation (#2111) ... * Make private toots get PuSHed to subscription URLs that belong to domains where you have approved followers * Authorized followers controller, stub for bulk action * Soft block in the background * Add simple test for new controller * Rename Settings::FollowersController to Settings::FollowerDomainsController, paginate results, rename "private" post setting to "followers-only", fix pagination style, improve post privacy preferences style, improve warning style * Extract compose form warnings into own container, show warning when posting to followers-only with unlocked account	8 years ago
Matt Jankowski	c44a700252	Quick best practice cleanup of views/helpers (#1546) ... * Remove trailing whitespace * Use query methods instead of explicit .blank? checks	8 years ago
Matt Jankowski	7f0a865b05	Allow import/export of mutes list (#1541) ... * Allow export of mutes list * Allow importing of mutes list * Refactor to use Settings::Exports::BaseController and DRY up exports code	8 years ago
Matt Jankowski	89e8e110c8	Imports controller errors (#1553) ... * Add spec for settings/imports controller * Add failing spec for settings/imports#create * Fix broken imports * Refactor ImportWorker	8 years ago
Jantso Porali	e30bbb1cb0	Fix for Import issue from URL (#1416)	8 years ago
Eugen	982fef811e	Fix #1141, fix #1126 - Avatar/profile info fetching (#1215) ... * Fix #1141, fix #1126 - Work through UpdateRemoteProfileService for both <feed> and <entry> top-level tags * Improve code quality, remove line unrelated to fix	8 years ago
Eugen	6d6a429af8	Rewrite Atom generation from stream entries to use Ox instead of Nokogiri (#1124) ... * Rewrite Atom generation from stream entries to use Ox instead of Nokogiri::Builder StreamEntry is now limited to only statuses, which allows some optimization. Removed extra queries on AccountsController#show. AtomSerializer instead of AtomBuilderHelper used in AccountsController#show, StreamEntriesController#show, StreamEntryRenderer and PubSubHubbub::DistributionWorker PubSubHubbub::DistributionWorker moves n+1 DomainBlock query to PubSubHubbub::DeliveryWorker instead. All Salmon slaps that aren't based on StreamEntry still use AtomBuilderHelper and Nokogiri * All Salmon slaps now use Ox instead of Nokogiri. No touch from status on account	8 years ago
blackle	540d6efe88	Catch more errors in process_follows so it doesn't fail	8 years ago
Eugen Rochko	5442083b3c	Split SalmonWorker into smaller parts, move profile updating into another job	8 years ago
Eugen Rochko	5b95be1c42	Replace calls to FeedManager#inline_render and #broadcast	8 years ago
Eugen	c9ebd5d19f	Fix wrong variable used in publish channel	8 years ago
Eugen	1b8c244dff	Add proper message to PushUpdateWorker, use redis directly	8 years ago
Eugen	3d8b80e1cc	Merge branch 'master' into broadcast-to-worker	8 years ago
Eugen Rochko	220bc48e8e	Only render public payload once in FanOutOnWrite	8 years ago
Kurtis Rainbolt-Greene	22dcadedb4	We're going to want these nice helper methods, lets share them with a parent class that matches Rails 5 practices (application level abstraction)	8 years ago
Kurtis Rainbolt-Greene	9638894233	Moving in the inline render	8 years ago
Kurtis Rainbolt-Greene	220051b8b2	I don't actually think we need that.	8 years ago
Kurtis Rainbolt-Greene	0069c01285	Moving the queue_at into the worker	8 years ago
Kurtis Rainbolt-Greene	96ef933820	Replacing the broadcast method with the one defined in the feed manager	8 years ago
Kurtis Rainbolt-Greene	1e96ce378e	By pushing this into a worker we can reduce the amount of time the feed manager using workers eat up a connection	8 years ago
Eugen Rochko	bda37489ac	Remove PuSH subscriptions when delivery is answered with a 4xx error	8 years ago
Eugen Rochko	6fd865c000	Spawn FeedInsertWorker to deliver status into personal feed	8 years ago
Eugen Rochko	82aaedec46	Reduce number of items in feeds, optimize regeneration worker slightly, ... make regeneration worker unique, (only schedule/execute once at a time)	8 years ago
Eugen Rochko	b21f7c28f6	Move OStatus processing back into default queue	8 years ago
Eugen Rochko	b510a56c0c	Only call regeneration worker after first login after a 14 day break	8 years ago
Eugen Rochko	f722bd2387	Separate background jobs into different queues. ATTENTION: new queue "pull" ... must be added to the Sidekiq invokation in your systemd file The pull queue will handle link crawling, thread resolving, and OStatus processing. Such tasks are more likely to hang for a longer time (due to network requests) so it is more sensible to not make the "in-house" tasks wait for them.	8 years ago
Eugen Rochko	b7c1b12367	Make default admin UI page reports. Add admin UI for creating a domain block	8 years ago
Eugen Rochko	e8875c6046	Import feature for following/blocking lists (addresses #62, #177, #201, #454)	8 years ago
Eugen Rochko	6b81d10030	Add digest e-mails	8 years ago
Eugen Rochko	40a4053732	Disable PuSH for blocked domains	8 years ago
Eugen Rochko	720ff55262	Adding more unit tests. Fixing Salmon slaps XML	8 years ago
Eugen Rochko	db814543c0	Take out private toot distribution out of PuSH again	8 years ago
Eugen Rochko	446aad4ce2	Make PuSH only distribute to subscriptions that match follower domains ... Allow PuSH to distribute private toots with that condition	8 years ago
Eugen Rochko	00b5731ecb	After FollowService, re-fetch remote account asynchronously, do nothing ... if account lock info was up to date, otherwise re-do the FollowService with now updated information	8 years ago
Eugen Rochko	149887a0ff	Make follow requests federate	8 years ago
Eugen Rochko	19b9e1e2c3	Preheat status cache	8 years ago
Eugen Rochko	3f075c7794	API for apps to register for push notifications	8 years ago
Eugen Rochko	8a880a3d46	Make blocks create entries and unfollows instantly, but do the clean up ... in the background instead. Should fix delay where blocked person can interact with blocker for a short time before background job gets processed	8 years ago
Eugen Rochko	cca82bf0a2	Move merging/unmerging of timelines into background. Move blocking into ... background as well since it's a computationally expensive	8 years ago
Eugen Rochko	f0de621e76	Fix #463 - Fetch and display previews of URLs using OpenGraph tags	8 years ago
Eugen Rochko	6c28886317	Improve background jobs params and error handling	8 years ago
Eugen Rochko	86be6d48c9	Fix accidental distribution of hidden stream entries to PuSH subscribers	8 years ago
Eugen Rochko	49834a6e7f	Add API for retrieving blocked accounts	8 years ago
Eugen Rochko	4c6809f6ab	Adding some rescues to workers, e.g. don't fail removal job when status can't be found in the first place (already removed)	8 years ago
Eugen Rochko	0542773bca	Make unfavouriting async to prevent timeout errors from leaving orphaned records behind	8 years ago
Eugen Rochko	6de079a5af	Removing external hub completely, fix #333 fixing digit-only hashtags, ... removing web app capability from non-webapp pages	8 years ago
Eugen Rochko	668013265c	Restoring old async behaviour of thread resolving as it proved to be more robust	8 years ago
Eugen Rochko	f90133d2ad	Thread resolving no longer needs to be separate from ProcessFeedService, ... since that is only ever called in the background	8 years ago
Eugen Rochko	f406e01fcf	Add filters for suspended accounts	8 years ago
Eugen Rochko	58b3f4fd67	Fix #329 - avatar errors no longer prevent remote accounts from being saved ... (without avatar). Also improved search position of exact matches	8 years ago
Eugen Rochko	4a2347da41	Track successful PuSH deliveries	8 years ago
Eugen Rochko	93a90cd9c3	Delete statuses asynchronously but provide instant feedback in the API	8 years ago
Eugen Rochko	cc70f28f19	Adding rack timeout of 30sec, PuSH jobs moved to push queue so they ... can be processed separately	8 years ago
Eugen Rochko	4e351baf88	Fix URLs in inline-rendered XML	8 years ago
Eugen Rochko	3ba6531611	Fix setting of confirmed=true on successful confirmation	8 years ago