441 Commits (67eb47e25f1e7704dd9ecd32695ff3b53c807375)

Author	SHA1	Message	Date
Eugen Rochko	9311430ed7	Prevent multiple handlers for Delete of Actor from running (#9292)	6 years ago
Eugen Rochko	8069fd636b	Remove intermediary arrays when creating hash maps from results (#9291)	6 years ago
ThibG	7f5a4be580	Fix emoji update date processing (#9255)	6 years ago
Eugen Rochko	4615512285	Reduce connect timeout limit and limit signature failures by source IP (#9236) ... * Reduce connect timeout from 10s to 1s * Limit failing signature verifications per source IP	6 years ago
Eugen Rochko	be202f9377	Accept the same payload in multiple inboxes and deliver (#9150)	6 years ago
m.b	2b18f5f85d	Add Page AP type support (#9121)	6 years ago
ThibG	e53cc673e7	Ignore invalid hashtags on remote statuses instead of rejecting them (#9118) ... Fixes #9115	6 years ago
ThibG	82e7988afc	Fix missing `mention` argument when processing incoming Create activities (#9114) ... * Fix missing `mention` argument when processing incoming Create activities * Fix typo (param → params)	6 years ago
Eugen Rochko	d4cf963749	Allow inbox owner to view implicitly targeted ActivityPub payload (#9093) ... Fix #9091	6 years ago
Eugen Rochko	fd5285658f	Add option to block reports from domain (#8830)	6 years ago
Eugen Rochko	ddd30f331c	Improve support for aspects/circles (#8950) ... * Add silent column to mentions * Save silent mentions in ActivityPub Create handler and optimize it Move networking calls out of the database transaction * Add "limited" visibility level masked as "private" in the API Unlike DMs, limited statuses are pushed into home feeds. The access control rules between direct and limited statuses is almost the same, except for counter and conversation logic * Ensure silent column is non-null, add spec * Ensure filters don't check silent mentions for blocks/mutes As those are "this person is also allowed to see" rather than "this person is involved", therefore does not warrant filtering * Clean up code * Use Status#active_mentions to limit returned mentions * Fix code style issues * Use Status#active_mentions in Notification And remove stream_entry eager-loading from Notification	6 years ago
Eugen Rochko	21ad21cb50	Improve signature verification safeguards (#8959) ... * Downcase signed_headers string before building the signed string The HTTP Signatures draft does not mandate the “headers” field to be downcased, but mandates the header field names to be downcased in the signed string, which means that prior to this patch, Mastodon could fail to process signatures from some compliant clients. It also means that it would not actually check the Digest of non-compliant clients that wouldn't use a lowercased Digest field name. Thankfully, I don't know of any such client. * Revert "Remove dead code (#8919)" This reverts commit a00ce8c92c. * Restore time window checking, change it to 12 hours By checking the Date header, we can prevent replaying old vulnerable signatures. The focus is to prevent replaying old vulnerable requests from software that has been fixed in the meantime, so a somewhat long window should be fine and accounts for timezone misconfiguration. * Escape users' URLs when formatting them Fixes possible HTML injection * Escape all string interpolations in Formatter class Slightly improve performance by reducing class allocations from repeated Formatter#encode calls * Fix code style issues	6 years ago
Eugen Rochko	61d44dd11f	Fix typo in ActivityPub Create handler (#8952) ... Regression from #8951	6 years ago
Eugen Rochko	790d3bc637	Move network calls out of transaction in ActivityPub handler (#8951) ... Mention and emoji code may perform network calls, but does not need to do that inside the database transaction. This may improve availability of database connections when using pgBouncer in transaction mode.	6 years ago
Eugen Rochko	774ac47373	Add conversations API (#8832) ... * Add conversations API * Add web UI for conversations * Add test for conversations API * Add tests for ConversationAccount * Improve web UI * Rename ConversationAccount to AccountConversation * Remove conversations on block and mute * Change last_status_id to be a denormalization of status_ids * Add optimistic locking	6 years ago
Jeong Arm	144d73730d	Leave unknown language as nil if account is remote (#8861) ... * Force use language detector if account is remote * Set unknown remote toot's language as nil	6 years ago
ThibG	ba444797d2	Fix handling of ActivityPub activities lacking some attributes (#8864)	6 years ago
Eugen Rochko	ef69c655cc	Fix class autoloading issue in ActivityPub::Activity::Create (#8820)	6 years ago
cbayerlein	4b78546135	Exclude replies from list timelines (#8683) ... * Changed list behaviour I added the following line to the FeedManager (app/lib/feed_manager.rb) in the push_to_list function: `return false if status.reply?` Now all posts that are replies are filtered out, so that now only "genuine" posts are displayed in the list. This is a first approach to solve issue #5916 * Update feed_manager.rb As suggested by @Gargron	6 years ago
ふぁぼ原	f7a6f9489d	Add a new preference to always hide all media (#8569)	6 years ago
Matt Sweetman	1889526e23	Add user preference to always expand toots marked with content warnings (#8762)	6 years ago
Eugen Rochko	f4d549d300	Redesign forms, verify link ownership with rel="me" (#8703) ... * Verify link ownership with rel="me" * Add explanation about verification to UI * Perform link verifications * Add click-to-copy widget for verification HTML * Redesign edit profile page * Redesign forms * Improve responsive design of settings pages * Restore landing page sign-up form * Fix typo * Support <link> tags, add spec * Fix links not being verified on first discovery and passive updates	6 years ago
Renato "Lond" Cerqueira	fe56d26f7b	Fix autoplay issue with spoiler tag (#8540) ... Add tests to avoid similar issues in the future	7 years ago
Eugen Rochko	e3764bdb52	Do not sign useless User-Agent or Accept-Encoding headers (#8533) ... Fix #8080	7 years ago
Renato "Lond" Cerqueira	11658d8653	Add animate custom emoji param to embed pages (#8507) ... * Add animate custom emoji param to embed pages * Rename param, use it for avatars and gifs * Fix issues pointed by codeclimate and breaking test * Ignore brakeman warning	7 years ago
Eugen Rochko	cabdbb7f9c	Add CLI task for rotating keys (#8466) ... * If an Update is signed with known key, skip re-following procedure Because it means the remote actor did *not* lose their database * Add CLI method for rotating keys bin/tootctl accounts rotate [USERNAME] Generates a new RSA key per account and sends out an Update activity signed with the old key. * Key rotation: Space out Update fan-outs every 5 minutes per 1000 accounts * Skip suspended accounts in key rotation	7 years ago
Quint Guvernator	da13fa5021	Fix low-hanging rubocop gripes (#8458) ... * rubocop: quit being so picky * rubocop: miscellany * rubocop: prefer present to blank	7 years ago
Eugen Rochko	b4ba4b1b5d	Spread out crawling randomly to avoid DDoSing the link (#8445) ... * Spread out crawling randomly to avoid DDoSing the link Fix #4486 * Remove trailing whitespace	7 years ago
M Somerville	95bd0d4528	Support ActivityStreams’ summaryMap. (#8422) ... In the same way as contentMap and nameMap.	7 years ago
Jakub Mendyk	6cb3514d64	Add ability to change an instance default theme from the administration panel (#7092) (#8381) ... * Add default_settings class method to ScopedSettings ScopedSettings was extended to use value of unscoped setting instead of only using defaults set in config/settings.yml for selected settings. This adds possibility for admins to set default values of users' settings, for example default theme (as requested in #7092). * Add ability to change an instance default theme Closes #7092	7 years ago
ThibG	42573b76f1	Do not crash if remote custom emoji does not define updated date (fixes #8376) (#8377)	7 years ago
Eugen Rochko	802cf6a4c5	Improve federated ID validation (#8372) ... * Fix URI not being sufficiently validated with prefetched JSON * Add additional id validation to OStatus documents, when possible	7 years ago
ThibG	59f7f4c923	Implement Undo { Accept { Follow } } (fixes #8234) (#8245) ... * Add Follow#revoke_request! * Implement Undo { Accept { Follow } } (fixes #8234)	7 years ago
ThibG	1ee675d68b	Use correct activity id in Accept when receiving duplicate Follow (fixes #8218) (#8244)	7 years ago
Eugen Rochko	39e361a56d	Expect relays to answer with accept/reject (#8179)	7 years ago
ThibG	c52bcc0331	Serialize text-less statuses as '.' over OStatus (fixes #7856) (#8126)	7 years ago
Eugen Rochko	07b799468d	Fix incorrect context definition for the "featured" keyword (#8090) ... * Fix incorrect context definition for the "featured" keyword Fix #8077 * Adjust context definition for movedTo	7 years ago
Eugen Rochko	a23ac107e4	Fix activity:interactions counter to count all interactions (#8037)	7 years ago
Eugen Rochko	8f64b17d98	Whitelist dat/ipfs/gopher links in sanitizer (#8034) ... Fix #7994	7 years ago
Eugen Rochko	2354b10eb5	Add admin dashboard (#8029)	7 years ago
Eugen Rochko	38e9662d78	Disable language detection for texts shorter than 140 characters (#8010) ... If the input text is blank after preparation (only mention, or only URL, or empty as in a media post), then use nil as language, since it's OK to show to everyone. Otherwise, always fall back to the server's default locale	7 years ago
Eugen Rochko	47c59591e0	Do not count self in potential friendships (#8014)	7 years ago
Eugen Rochko	20fefdb714	Make whole-word filter regex consistent between Ruby and JS (#7987)	7 years ago
ThibG	1ca4e51eb3	Add option to not consider word boundaries when processing keyword filtering (#7975) ... * Add option to not consider word boundaries when filtering phrases * Add a few tests for keyword/phrase filtering	7 years ago
Eugen Rochko	404c7702ec	In keyword filter, account for reblogs, HTML and whole-words (#7960) ... * In keyword filter, account for reblogs, HTML and whole-words * Match whole words in JS filter, too * Fix typo	7 years ago
Eugen Rochko	4b198b172d	Check reblogged status for blocked/muted mentions (#7957)	7 years ago
Eugen Rochko	da8fe8079e	Re-add follow recommendations API (#7918) ... * Re-add follow recommendations API GET /api/v1/suggestions Removed in 8efa081f21 due to Neo4J dependency. The algorithm uses triadic closures, takes into account suspensions, blocks, mutes, domain blocks, excludes locked and moved accounts, and prefers more recently updated accounts. * Track interactions with people you don't follow Replying to, favouriting and reblogging someone you're not following will make them show up in follow recommendations. The interactions have different weights: - Replying is 1 - Favouriting is 10 (decidedly positive interaction, but private) - Reblogging is 20 Following them, muting or blocking will remove them from the list, obviously. * Remove triadic closures, ensure potential friendships are trimmed	7 years ago
MIYAGI Hikaru	ddd0bb69e1	Merge `HIDDEN_SERVICE_VIA_TRANSPARENT_PROXY` into `ALLOW_ACCESS_TO_HIDDEN_SERVICE` (#7901) ... If Mastodon accesses to the hidden service via transparent proxy, it's needed to avoid checking whether it's a private address, since `.onion` is resolved to a private address. I was previously using the `HIDDEN_SERVICE_VIA_TRANSPARENT_PROXY` to provide that function. However, I realized that using `HIDDEN_SERVICE_VIA_TRANSPARENT_PROXY` is redundant, since this specification is always used with `ALLOW_ACCESS_TO_HIDDEN_SERVICE`. Therefore, I decided to integrate the setting of `HIDDEN_SERVICE_VIA_TRANSPARENT_PROXY` into` ALLOW_ACCESS_TO_HIDDEN_SERVICE`.	7 years ago
Eugen Rochko	cdb101340a	Keyword/phrase filtering (#7905) ... * Add keyword filtering GET|POST /api/v1/filters GET|PUT|DELETE /api/v1/filters/:id - Irreversible filters can drop toots from home or notifications - Other filters can hide toots through the client app - Filters use a phrase valid in particular contexts, expiration * Make sure expired filters don't get applied client-side * Add missing API methods * Remove "regex filter" from column settings * Add tests * Add test for FeedManager * Add CustomFilter test * Add UI for managing filters * Add streaming API event to allow syncing filters * Fix tests	7 years ago
Eugen Rochko	a58ec29631	Allow selecting default posting language instead of auto-detect (#7828) ... * Allow selecting default posting language instead of auto-detect * Enable default language setting in credentials API * Fix form saving	7 years ago
Eugen Rochko	6ba7d9d0d9	Do not accept ActivityPub follow requests from blocked user (#7756) ... * Do not accept ActivityPub follow requests from blocked user Fix #7745 * Deliver auto-rejection immediately when follow-requested by blocked account * Fix trailing whitespace	7 years ago
Eugen Rochko	1e938b966e	Exclude unlisted, private and direct toots from affecting trends (#7686)	7 years ago
abcang	90908fc24b	Fix N+1 on AtomSerializer (#7669)	7 years ago
Eugen Rochko	ab36e0ef72	Record trending tags from ActivityPub, too (#7647)	7 years ago
ThibG	cdbdf7f98b	Ignore multiple occurrences of a hashtag within a status (fixes #7585) (#7606)	7 years ago
Eugen Rochko	57b503d4ef	Resolve unknown status from Add activity, skip Remove if unknown (#7526) ... Fix #7518	7 years ago
Eugen Rochko	1e02dc8715	Add preference to hide following/followers lists (#7532) ... * Add preference to hide following/followers lists - Public pages - ActivityPub collections (does not return pages but does give total) - REST API (unless it's your own) (does not federate) Fix #6901 * Add preference * Add delegation * Fix issue * Fix issue	7 years ago
MIYAGI Hikaru	919eef3098	User agent for WebFinger (#7531) ... * User agent for WebFinger * local_domain → web_domain * 'http' is away accidentally...	7 years ago
Akihiko Odaki	55fd55714a	Raise Mastodon::RaceConditionError if Redis lock failed (#7511) ... An explicit error allows user agents to know the error and Sidekiq to retry.	7 years ago
ThibG	7467361d70	Fetch boosted statuses on behalf of a follower (fixes #7426) (#7459) ... When an ActivityPub Announce is processed and the boosted toot is not known, fetch it on behalf of one of the booster's followers. This is to allow fetching self-boosts of previously-unknown private toots. If fetching on behalf of a user fails, try fetching it anonymously: the selected follower of a boosting user may be banned by the boosted toot's author.	7 years ago
Yamagishi Kazutoshi	6588f6a0a9	Disable inflate gzip on head method (#7432)	7 years ago
Yamagishi Kazutoshi	edf882320a	Support gzip encoding on Request (#7425)	7 years ago
Eugen Rochko	6208ea5a53	If an OStatus message contains nsfw hashtag, mark it as sensitive (#7398) ... * If an OStatus message contains nsfw hashtag, mark it as sensitive Undo parts of #7048 * Put nsfw hashtag on OStatus messages if they have any media * Fix code style issues	7 years ago
Eugen Rochko	61a9018607	Enable custom emojis in profiles (notes, field values, display names) (#7374) ... Follow-up to #6124	7 years ago
Eugen Rochko	c947e2e4c5	Fix handling of malformed ActivityPub payloads when URIs are nil (#7370) ... * Fix handling of malformed ActivityPub payloads when URIs are nil * Gracefully handle JSON-LD canonicalization failures	7 years ago
Eugen Rochko	6793bec4c6	Store URIs of follows, follow requests and blocks for ActivityPub (#7160) ... Same URI passed between follow request and follow, since they are the same thing in ActivityPub. Local URIs are generated during creation using UUIDs and are passed to serializers.	7 years ago
ThibG	a24605961a	Fixes/do not override timestamps (#7336) ... * Revert "Fixes/do not override timestamps (#7331)" This reverts commit 581a5c9d29. * Document Snowflake ID corner-case a bit more Snowflake IDs are used for two purposes: making object identifiers harder to guess and ensuring they are in chronological order. For this reason, they are based on the `created_at` attribute of the object. Unfortunately, inserting items with older snowflakes IDs will break the assumption of consumers of the paging APIs that new items will always have a greater identifier than the last seen one. * Add `override_timestamps` virtual attribute to not correlate snowflake ID with created_at	7 years ago
ThibG	581a5c9d29	Fixes/do not override timestamps (#7331) ... * Do not override timestamps for incoming toots * Remove every reference to override_timestamps Statuses are now created with the announced publishing date and are only pushed to timelines if that date is at most 6 hours earlier than the time at which it is processed.	7 years ago
Eugen Rochko	a5293fdf61	Fix n+1 queries in StatusThreadingConcern (#7321)	7 years ago
Eugen Rochko	cb5b5cb5f7	Slightly reduce RAM usage (#7301) ... * No need to re-require sidekiq plugins, they are required via Gemfile * Add derailed_benchmarks tool, no need to require TTY gems in Gemfile * Replace ruby-oembed with FetchOEmbedService Reduce startup by 45382 allocated objects * Remove preloaded JSON-LD in favour of caching HTTP responses Reduce boot RAM by about 6 MiB * Fix tests * Fix test suite by stubbing out JSON-LD contexts	7 years ago
ThibG	d0cdd5cf94	Accept actor object updates from all supported actor types (#7312)	7 years ago
Eugen Rochko	f62539ce5c	Remove most behaviour disparities between blocks and mutes (#7231) ... * Remove most behaviour disparities between blocks and mutes The only differences between block and mute should be: - Mutes can optionally NOT affect notifications - Mutes should not be visible to the muted Fix #7230 Fix #5713 * Do not allow boosting someone you blocked Fix #7248 * Do not allow favouriting someone you blocked * Fix nil error in StatusPolicy	7 years ago
Eugen Rochko	965345316f	Guard against nil URLs in Request class (#7284) ... Fix #7265	7 years ago
Surinna Curtis	dc786c0cf4	Support Actors/Statuses with multiple types (#7305) ... * Add equals_or_includes_any? helper in JsonLdHelper * Support arrays in JSON-LD type fields for actors/tags/objects. * Spec for resolving accounts with extension types * Style tweaks for codeclimate	7 years ago
Eugen Rochko	705f1d7bf1	Fix missing updated_at attribute on emoji EntityCache (#7297) ... Just don't try to save space by only selecting few attributes. If anyone is wondering, this is needed because the emoji entity cache is not really only used for entities, it's accessed again to generate Emoji tags in ActivityPub/OStatus, so a lot more properties are used than what is needed in HTML alone...	7 years ago
Eugen Rochko	295e3ef02b	Fix missing domain attribute in EntityCache for emoji (#7290)	7 years ago
Eugen Rochko	a872392cd9	Add entity cache (#7271) ... * Add entity cache Use a caching layer for mentions and custom emojis that are dynamically extracted from text. Reduce duplicate text extractions * Fix code style issue	7 years ago
MIYAGI Hikaru	f58dcbc981	HTTP proxy support for outgoing request, manage access to hidden service (#7134) ... * Add support for HTTP client proxy * Add access control for darknet Supress error when access to darknet via transparent proxy * Fix the codes pointed out * Lint * Fix an omission + lint * any? -> include? * Change detection method to regexp to avoid test fail	7 years ago
Eugen Rochko	9d4710ed00	Add RSS feeds for end-users (#7259) ... * Add RSS feed for accounts * Add RSS feeds for hashtags * Fix code style issues * Fix code style issues	7 years ago
Eugen Rochko	7db7d68136	Detect and prevent image bombs, max. processable dimension 4096^2 (#7229)	7 years ago
Eugen Rochko	75c4ab9d12	Remove "nsfw" category for sensitive statuses in OStatus serializer (#7048) ... Fix #7011	7 years ago
Eugen Rochko	78ed4ab75f	Add bio fields (#6645) ... * Add bio fields - Fix #3211 - Fix #232 - Fix #121 * Display bio fields in web UI * Fix output of links and missing fields * Federate bio fields over ActivityPub as PropertyValue * Improve how the fields are stored, add to Edit profile form * Add rel=me to links in fields Fix #121	7 years ago
Levi Bard	cd0eaa349c	Enable updating additional account information from user preferences via rest api (#6789) ... * Enable updating additional account information from user preferences via rest api Resolves #6553 * Pacify rubocop * Decoerce incoming settings in UserSettingsDecorator * Create user preferences hash directly from incoming credentials instead of going through ActionController::Parameters * Clean up user preferences update * Use ActiveModel::Type::Boolean instead of manually checking stringified number equivalence	7 years ago
Eugen Rochko	b5726def55	Forward deletes on the same path as reply forwarding (#7058) ... * Forward deletes on the same path as reply forwarding * Remove trailing whitespace	7 years ago
David Underwood	123a343d11	[WIP] Enable custom emoji on account pages and in the sidebar (#6124) ... Federate custom emojis with accounts	7 years ago
ThibG	e573bb0990	Fix compatibility with PeerTube (#6968) ... * Support fetching objects of convertible types by URL (fixes #6924) * Ignore invalid hashtags	7 years ago
ThibG	9ed5eebd7c	Do not ignore unknown media attachments, only skip them (#6948) ... That way, they are displayed in a list below the corresponding toot.	7 years ago
Akihiko Odaki	40e5d2303b	Validate HTTP response length while receiving (#6891) ... to_s method of HTTP::Response keeps blocking while it receives the whole content, no matter how it is big. This means it may waste time to receive unacceptably large files. It may also consume memory and disk in the process. This solves the inefficency by checking response length while receiving.	7 years ago
Akihiko Odaki	54b273bf99	Close http connection in perform method of Request class (#6889) ... HTTP connections must be explicitly closed in many cases, and letting perform method close connections makes its callers less redundant and prevent them from forgetting to close connections.	7 years ago
Rey Tucker	36b5703796	request: in the event of failure, try other IPs (#6761) (#6813) ... * request: in the event of failure, try other IPs (#6761) In the case where a name has multiple A/AAAA records, we should try subsequent records instead of immediately failing when we have a failure on the first IP address. This significantly improves delivery success when there are network connectivity problems affecting only IPv4 or IPv6. * fix method call style * request_spec: adjust test case to use Addrinfo * request: Request/open: move private addr check to within begin/rescue * request_spec: add case to test failover, fix exception check * Double Addrinfo.foreach so that it correctly yields instances	7 years ago
ThibG	ff6b8a6443	Serialize mentions in the order they are added (#6836) ... Up until now, the order seemed to be in the *opposite* order, which caused the WebUI to populate mentions in reversed order when replying to toots local to one's instance.	7 years ago
Eugen Rochko	f02411da40	Ignore media validation when attaching to status during processing (#6822) ... Fix #6821	7 years ago
MitarashiDango	6dcf96271e	fix validation error (media only status) (#6684) ... * fix validation error (media only status) * Incorporating review suggestions * Reflect similar fix to OStatus side * Fix not to include media in transaction * Restore the limit of the number of media * Fix not to return nil	7 years ago
Eugen Rochko	cfa9b6e13a	Remove text requirement when media attached from statuses (#6672)	7 years ago
Eugen Rochko	e6520c0270	Fix #6657 - Use target instead of origin in Remove activity (#6664)	7 years ago
Eugen Rochko	9110db41c5	Federate pinned statuses over ActivityPub (#6610) ... * Federate pinned statuses over ActivityPub * Display pinned toots in web UI Fix #6117 * Fix migration * Fix tests * Update outbox_serializer.rb * Update remove_serializer.rb * Update add_serializer.rb * Update fetch_featured_collection_service.rb	7 years ago
Eugen Rochko	e852872846	Fix #5708: Reject->Follow will remove the follow if it exists (#6571)	7 years ago
Eugen Rochko	41a01bec23	Federated reports (#6570) ... * Fix #2176: Federated reports * UI for federated reports * Add spec for ActivityPub Flag handler * Add spec for ReportService	7 years ago
Akihiko Odaki	2e8a492e88	Raise Mastodon::HostValidationError when host for HTTP request is private (#6410)	7 years ago
Akihiko Odaki	f8f0572ee0	Do not push status to feed if its reblog is already inserted (#6488) ... A complemental change for precompute_feed_service_spec.rb also fixes its random failure which is caused by the Snowlake randomization of the order of an original status and its reblog.	7 years ago