140 Commits (0153a239dbecd59df6b36c5469f85a340daf4dc1)

Author	SHA1	Message	Date
ThibG	720214feb0	Add support for inlined objects in activity audience (#14514) ... * Add support for inlined objects in activity audience * Add tests	5 years ago
ThibG	bcf85b5208	Dereference object URIs in Create and Update messages (#14359) ... * Dereference object URIs in Create and Update messages Fixes #14353 Signed-off-by: Thibaut Girka <thib@sitedethib.com> * Refactor, and perform origin check *before* attempting to fetch object Co-authored-by: Fire Demon <firedemon@creature.cafe>	5 years ago
abcang	6a96af4d20	Fix rubocop warning (#14288) ... * Fix rubocop warning * use limit variable * use ContextCreatingMethods option	5 years ago
Eugen Rochko	64aac30733	Add customizable thumbnails for audio and video attachments (#14145) ... - Change audio files to not be stripped of metadata - Automatically extract cover art from audio if it exists - Add `thumbnail` parameter to `POST /api/v1/media`, `POST /api/v2/media` and `PUT /api/v1/media/:id` - Add `icon` to represent it in attachments in ActivityPub - Fix `preview_url` containing URL of missing missing image when there is no thumbnail instead of null - Fix duration of audio not being displayed on public pages until the file is loaded	5 years ago
Eugen Rochko	5d8398c8b8	Add E2EE API (#13820)	6 years ago
ThibG	a319c1e60f	Add support for `summary` field for media description (#13763)	6 years ago
ThibG	2ee5a9d9c3	Clean up OStatus-related codepaths (#12173) ... * Remove “protocol” argument and return value, as only ActivityPub is supported * Remove FetchRemoteAccountService, only use ActivityPub::FetchRemoteAccountService * Fix tests	6 years ago
ThibG	da2143b308	Fixes featured hashtag setting page erroring out instead of rejecting invalid tags (#12436) ... * Revert "Fix ignoring whole status because of one invalid hashtag (#11621)" This reverts commit dff46b260b. * Fix statuses being rejected because of invalid hashtag names * Add spec for invalid hashtag names in statuses * Add test for featured tags controller	6 years ago
ThibG	f05b0463db	Fallback to Create audience when object has no defined audience (#12249) ... Fixes #11137	6 years ago
ThibG	3babf8464b	Add voters count support (#11917) ... * Add voters count to polls * Add ActivityPub serialization and parsing of voters count * Add support for voters count in WebUI * Move incrementation of voters count out of redis lock * Reword “voters” to “people”	6 years ago
Eugen Rochko	4f6af87906	Change spam check to apply to local accounts and add a threshold (#11806) ... Instead of detecting spam on first duplicate message, add a threshold of 5 such messages to reduce false positives	6 years ago
Eugen Rochko	031ca25014	Add retry for failed media downloads and `tootctl media refresh` (#11775)	6 years ago
Eugen Rochko	b9b0313c78	Revert "Remove conversation URI (#11423)" (#11424) ... This reverts commit 75f7f9930e.	6 years ago
Eugen Rochko	75f7f9930e	Remove conversation URI (#11423) ... It is not part of ActivityPub and will free up a lot of space	6 years ago
Eugen Rochko	f371b32137	Change hashtags to preserve first-used casing (#11416)	6 years ago
Eugen Rochko	6ff67be0f6	Add a spam check (#11217) ... * Add a spam check * Use Nilsimsa to generate locality-sensitive hashes and compare using Levenshtein distance * Add more tests * Add exemption when the message is a reply to something that mentions the sender * Use Nilsimsa Compare Value instead of Levenshtein distance * Use MD5 for messages shorter than 10 characters * Add message to automated report, do not add non-public statuses to automated report, add trust level to accounts and make unsilencing raise the trust level to prevent repeated spam checks on that account * Expire spam check data after 3 months * Add support for local statuses, reduce expiration to 1 week, always create a report * Add content warnings to the spam check and exempt empty statuses * Change Nilsimsa threshold to 95 and make sure removed statuses are removed from the spam check * Add all matched statuses into automatic report	6 years ago
Eugen Rochko	63c7fe8e48	Refactor controllers for statuses, accounts, and more (#11249)	6 years ago
Eugen Rochko	aa9b37822b	Fix audio not being downloaded from remote servers (#11145)	7 years ago
Eugen Rochko	707ddf7808	Change domain blocks to automatically support subdomains (#11138) ... * Change domain blocks to automatically support subdomains If a more authoritative domain is blocked (example.com), then the same block will be applied to a subdomain (foo.example.com) * Match subdomains of existing accounts when blocking/unblocking domains * Improve code style	7 years ago
ThibG	3a84bacf86	Handle blank poll options more gracefully (#10946) ... Pleroma currently allows (erroneously imho) empty poll options, that is, options with an empty (but existing) `name`.	7 years ago
ThibG	9a5561a5b8	Fix possible race condition when processing statuses (#10815)	7 years ago
Eugen Rochko	fba96c808d	Add blurhash (#10630) ... * Add blurhash * Use fallback color for spoiler when blurhash missing * Federate the blurhash and accept it as long as it's at most 5x5 * Display unknown media attachments as blurhash placeholders * Improve style of embed actions and spoiler button * Change blurhash resolution from 3x3 to 4x4 * Improve dependency definitions * Fix code style issues	7 years ago
Eugen Rochko	f1bc90ab50	Rename :poll to :preloadable_poll and :owned_poll to :poll on Status (#10401) ... Also, fix some n+1 queries Resolve #10365	7 years ago
Eugen Rochko	9f5b55ad4f	Fix poll update handler calling method was that was not available (#10246) ... * Fix poll update handler calling method was that was not available Fix regression from #10209 * Refactor VoteService * Refactor ActivityPub::DistributePollUpdateWorker and optimize it * Fix typo * Fix typo	7 years ago
ThibG	3a92885a86	Support pushing and receiving updates to poll tallies (#10209) ... * Process incoming poll tallies update * Send Update on poll vote * Do not send Updates for a poll more often than once every 3 minutes * Include voters in people to notify of results update * Schedule closing poll worker on poll creation * Add new notification type for ending polls * Add front-end support for ended poll notifications * Fix UpdatePollSerializer * Fix Updates not being triggered by local votes * Fix tests failure * Fix web push notifications for closing polls * Minor cleanup * Notify voters of both remote and local polls when those close * Fix delivery of poll updates to mentioned accounts and voters	7 years ago
ThibG	3aaac4f134	Do not allow adding votes to expired polls (#10214) ... * Do not allow adding votes to expired polls * Only validate expires_at on create	7 years ago
ThibG	34f261e6af	Avoid unnecessarily fetching the replies collection when it is empty (#10201)	7 years ago
ThibG	833ffce2df	Store remote votes URI (#10158) ... * Store remote votes URI * Add spec for accepting remote votes * Make poll vote id generation work the same way as follows	7 years ago
Eugen Rochko	0e6998da3c	Add tests for ActivityPub poll processing (#10143)	7 years ago
Eugen Rochko	ae1b9cf70a	Fix remote poll expiration time (#10144)	7 years ago
Eugen Rochko	230a012f00	Add polls (#10111) ... * Add polls Fix #1629 * Add tests * Fixes * Change API for creating polls * Use name instead of content for votes * Remove poll validation for remote polls * Add polls to public pages * When updating the poll, update options just in case they were changed * Fix public pages showing both poll and other media	7 years ago
ThibG	9d3c6f1849	Improved remote thread fetching (#10106) ... * Fetch up to 5 replies when discovering a new remote status This is used for resolving threads downwards. The originating server must add a “replies” attributes with such replies for it to be useful. * Add some tests for ActivityPub::FetchRepliesWorker * Add specs for ActivityPub::FetchRepliesService * Serialize up to 5 public self-replies for ActivityPub notes * Add specs for ActivityPub::NoteSerializer * Move exponential backoff logic to a worker concern * Fetch first page of paginated collections when fetching thread replies * Add specs for paginated collections in replies * Move Note replies serialization to a first CollectionPage The collection isn't actually paginable yet as it has no id nor a `next` field. This may come in another PR. * Use pluck(:uri) instead of map(&:uri) to improve performances * Fix fetching replies when they are in a CollectionPage	7 years ago
ThibG	9edab46368	Fix mention processing for unknwon accounts on incoming ActivityPub Notes (#10125) ... `::FetchRemoteAccountService` is not `ActivityPub::FetchRemoteAccountService`, its second argument is the pre-fetched body. Passing `id: false` actually passed a `Hash` as the prefetched body, instead of properly resolving unknown remote accounts.	7 years ago
Eugen Rochko	147b4c2c3a	Add logging for rejected ActivityPub payloads and add tests (#10062)	7 years ago
Eugen Rochko	c417e8c198	Filter incoming Announce activities by relation to local activity (#10041) ... * Filter incoming Announce activities by relation to local activity Reject if announcer is not followed by local accounts, and is not from an enabled relay, and the object is not a local status Follow-up to #10005 * Fix tests	7 years ago
Eugen Rochko	dad339da6d	Filter incoming Create activities by relation to local activity (#10005) ... Reject those from accounts with no local followers, from relays that are not enabled, which do not address local accounts and are not replies to accounts that do have local followers	7 years ago
ThibG	6a5307a573	Alternative handling of private self-boosts (#9998) ... * When self-boosting, embed original toot into Announce serialization * Process unknown self-boosts from Announce object if it is more than an URI * Add some self-boost specs * Only serialize private toots in self-Announces	7 years ago
ThibG	75b1488cf4	Add tombstones for remote statuses (#9830) ... * Add Tombstone model to remember object deletion * Do not recreate a status if it has been deleted * Record Tombstone for remote deleted items Also, only record deleted items from same-host actors * Clear an user's tombstones when their key change	7 years ago
Eugen Rochko	4ab42287c0	Use summary as summary for converted ActivityPub objects (#9823) ... Fix #8609	7 years ago
ThibG	a4f07bad95	Reduce chances of race conditions when processing deleted toots (#9815) ... * Reduce chances of race conditions when processing deleted toots * Prevent race condition when processing deleted toots	7 years ago
Eugen Rochko	aa9a20cde0	Fix ThreadResolveWorker getting queued with invalid URLs (#9628)	7 years ago
ThibG	7f5a4be580	Fix emoji update date processing (#9255)	7 years ago
Eugen Rochko	be202f9377	Accept the same payload in multiple inboxes and deliver (#9150)	7 years ago
m.b	2b18f5f85d	Add Page AP type support (#9121)	7 years ago
ThibG	e53cc673e7	Ignore invalid hashtags on remote statuses instead of rejecting them (#9118) ... Fixes #9115	7 years ago
ThibG	82e7988afc	Fix missing `mention` argument when processing incoming Create activities (#9114) ... * Fix missing `mention` argument when processing incoming Create activities * Fix typo (param → params)	7 years ago
Eugen Rochko	d4cf963749	Allow inbox owner to view implicitly targeted ActivityPub payload (#9093) ... Fix #9091	7 years ago
Eugen Rochko	ddd30f331c	Improve support for aspects/circles (#8950) ... * Add silent column to mentions * Save silent mentions in ActivityPub Create handler and optimize it Move networking calls out of the database transaction * Add "limited" visibility level masked as "private" in the API Unlike DMs, limited statuses are pushed into home feeds. The access control rules between direct and limited statuses is almost the same, except for counter and conversation logic * Ensure silent column is non-null, add spec * Ensure filters don't check silent mentions for blocks/mutes As those are "this person is also allowed to see" rather than "this person is involved", therefore does not warrant filtering * Clean up code * Use Status#active_mentions to limit returned mentions * Fix code style issues * Use Status#active_mentions in Notification And remove stream_entry eager-loading from Notification	7 years ago
Eugen Rochko	61d44dd11f	Fix typo in ActivityPub Create handler (#8952) ... Regression from #8951	7 years ago
Eugen Rochko	790d3bc637	Move network calls out of transaction in ActivityPub handler (#8951) ... Mention and emoji code may perform network calls, but does not need to do that inside the database transaction. This may improve availability of database connections when using pgBouncer in transaction mode.	7 years ago
Eugen Rochko	ef69c655cc	Fix class autoloading issue in ActivityPub::Activity::Create (#8820)	7 years ago
M Somerville	95bd0d4528	Support ActivityStreams’ summaryMap. (#8422) ... In the same way as contentMap and nameMap.	7 years ago
ThibG	42573b76f1	Do not crash if remote custom emoji does not define updated date (fixes #8376) (#8377)	7 years ago
Eugen Rochko	1e938b966e	Exclude unlisted, private and direct toots from affecting trends (#7686)	8 years ago
Eugen Rochko	ab36e0ef72	Record trending tags from ActivityPub, too (#7647)	8 years ago
ThibG	cdbdf7f98b	Ignore multiple occurrences of a hashtag within a status (fixes #7585) (#7606)	8 years ago
Akihiko Odaki	55fd55714a	Raise Mastodon::RaceConditionError if Redis lock failed (#7511) ... An explicit error allows user agents to know the error and Sidekiq to retry.	8 years ago
ThibG	a24605961a	Fixes/do not override timestamps (#7336) ... * Revert "Fixes/do not override timestamps (#7331)" This reverts commit 581a5c9d29. * Document Snowflake ID corner-case a bit more Snowflake IDs are used for two purposes: making object identifiers harder to guess and ensuring they are in chronological order. For this reason, they are based on the `created_at` attribute of the object. Unfortunately, inserting items with older snowflakes IDs will break the assumption of consumers of the paging APIs that new items will always have a greater identifier than the last seen one. * Add `override_timestamps` virtual attribute to not correlate snowflake ID with created_at	8 years ago
ThibG	581a5c9d29	Fixes/do not override timestamps (#7331) ... * Do not override timestamps for incoming toots * Remove every reference to override_timestamps Statuses are now created with the announced publishing date and are only pushed to timelines if that date is at most 6 hours earlier than the time at which it is processed.	8 years ago
Surinna Curtis	dc786c0cf4	Support Actors/Statuses with multiple types (#7305) ... * Add equals_or_includes_any? helper in JsonLdHelper * Support arrays in JSON-LD type fields for actors/tags/objects. * Spec for resolving accounts with extension types * Style tweaks for codeclimate	8 years ago
ThibG	e573bb0990	Fix compatibility with PeerTube (#6968) ... * Support fetching objects of convertible types by URL (fixes #6924) * Ignore invalid hashtags	8 years ago
ThibG	9ed5eebd7c	Do not ignore unknown media attachments, only skip them (#6948) ... That way, they are displayed in a list below the corresponding toot.	8 years ago
Eugen Rochko	f02411da40	Ignore media validation when attaching to status during processing (#6822) ... Fix #6821	8 years ago
MitarashiDango	6dcf96271e	fix validation error (media only status) (#6684) ... * fix validation error (media only status) * Incorporating review suggestions * Reflect similar fix to OStatus side * Fix not to include media in transaction * Restore the limit of the number of media * Fix not to return nil	8 years ago
Eugen Rochko	90f12f2e5a	Focal points (#6520) ... * Add focus param to media API, center thumbnails on focus point * Add UI for setting a focal point * Improve focal point icon on upload item * Use focal point in upload preview * Add focalPoint property to ActivityPub * Don't show focal point button for non-image attachments	8 years ago
Eugen Rochko	e4a241abef	Fix bad URL schemes being accepted (#6219) ... * Fix actors accepting invalid URI schemes or different host between URI and URL * Fix statuses accepting invalid URI scheme or different host to actor * Adjust tests to new requirements * Improve readability of mismatching_origin?/invalid_origin? methods	8 years ago
puckipedia	0eff42d688	Move Article from supported to converted types (#6218)	8 years ago
abcang	3caec1ecc2	Save media outside transaction (#5959)	8 years ago
Eugen Rochko	4c6b5dbe96	Add semi-support for Video/Image objects in ActivityPub (#5848) ... * Add semi-support for Video/Image objects in ActivityPub Video and Image objects will create corresponding status records with manually crafted text contents (title + URL) * Extract html-url-finding logic into JsonLdHelper * Fallback to id when url missing, extract supported object types	8 years ago
Eugen Rochko	85e97ecab6	Fix too many forwards (#5854) ... * Avoid sending explicit Undo->Announce when original deleted * Do not forward a reply back to the server that sent it * Deduplicate inboxes of rebloggers' followers for delete forwarding * Adjust test * Fix wrong class, bad SQL, wrong variable, outdated comment	8 years ago
Renato "Lond" Cerqueira	ad207456d6	Improve language filter (#5724) ... * Scrub text of html before detecting language. * Detect language on statuses coming from activitypub. * Fix rubocop comments. * Remove custom emoji from text before language detection	8 years ago
puckipedia	0cb329f63a	Allow ActivityPub Note's tag and attachment to be single objects (#5534)	8 years ago
Eugen Rochko	0717d9b3e6	Set snowflake IDs for backdated statuses (#5260) ... - Rename Mastodon::TimestampIds into Mastodon::Snowflake for clarity - Skip for statuses coming from inbox, aka delivered in real-time - Skip for statuses that claim to be from the future	8 years ago
Eugen Rochko	3a3475450e	Encode custom emojis as resolveable objects in ActivityPub (#5243) ... * Encode custom emojis as resolveable objects in ActivityPub * Improve code style	8 years ago
Akihiko Odaki	63f0979799	Validate id of ActivityPub representations (#5114) ... Additionally, ActivityPub::FetchRemoteStatusService no longer parses activities. OStatus::Activity::Creation no longer delegates to ActivityPub because the provided ActivityPub representations are not signed while OStatus representations are.	8 years ago
Eugen Rochko	4ec1771165	Add ability to specify alternative text for media attachments (#5123) ... * Fix #117 - Add ability to specify alternative text for media attachments - POST /api/v1/media accepts `description` straight away - PUT /api/v1/media/:id to update `description` (only for unattached ones) - Serialized as `name` of Document object in ActivityPub - Uploads form adjusted for better performance and description input * Add tests * Change undo button blend mode to difference	8 years ago
Akihiko Odaki	98936bfcdf	Add missing validations in ActivityPub::Activity::Create (#5096)	8 years ago
Akihiko Odaki	bb4d005a83	Introduce OStatus::TagManager (#5008)	8 years ago
Eugen Rochko	81cec35dbf	Custom emoji (#4988) ... * Custom emoji - In OStatus: `<link rel="emoji" name="coolcat" href="http://..." />` - In ActivityPub: `{ type: "Emoji", name: ":coolcat:", href: "http://..." }` - In REST API: Status object includes `emojis` array (`shortcode`, `url`) - Domain blocks with reject media stop emojis - Emoji file up to 50KB - Web UI handles custom emojis - Static pages render custom emojis as `<img />` tags Side effects: - Undo #4500 optimization, as I needed to modify it to restore shortcode handling in emojify() - Formatter#plaintext should now make sure stripped out line-breaks and paragraphs are replaced with newlines * Fix emoji at the start not being converted	8 years ago
ThibG	4a73615193	Fix race condition when receiving an ActivityPub Create multiple times (#4930) ... * Fix race condition when receiving an ActivityPub Create multiple times * Use a RedisLock to avoid concurrent processing of a same Create activity	8 years ago
Eugen Rochko	9b50a9dd83	Fix some ActivityPub JSON bugs (#4796) ... - Fix assumption that `url` is always a string. Handle it if it's an array of strings, array of objects, object, or string, both for accounts and for objects - `sharedInbox` is actually supposed to be under `endpoints`, handle both cases and adjust the serializer	8 years ago
Eugen Rochko	1b5806b744	Define missing JSON-LD properties (#4767) ... Using _: property names is discouraged, as in the future, canonicalization may throw an error when encountering that instead of discarding it silently like it does now. We are defining some ActivityStreams properties which we expect to land in ActivityStreams eventually, to ensure that future versions of Mastodon will remain compatible with this even once that happens. Those would be `locked`, `sensitive` and `Hashtag` We are defining a custom context inline for some properties which we do not expect to land in any other context. `atomUri`, `inReplyToAtomUri` and `conversation` are part of the custom defined OStatus context.	8 years ago
unarist	8fd8f81ae7	Deduplicate with local status on Create activity (#4763)	8 years ago
Eugen Rochko	7b8f262840	Forward ActivityPub creates that reply to local statuses (#4709) ... * Forward ActivityPub creates that reply to local statuses * Fix test * Fix wrong signers	8 years ago
Eugen Rochko	0d5d11eeff	Add _:inReplyToAtomUri to ActivityPub (#4702)	8 years ago
unarist	d63de55ef8	Fix bugs which OStatus accounts may detected as ActivityPub ready (#4662) ... * Fallback to OStatus in FetchAtomService * Skip activity+json link if that activity is Person without inbox * If unsupported activity was detected and all other URLs failed, retry with ActivityPub-less Accept header * Allow mention to OStatus account in ActivityPub * Don't update profile with inbox-less Person object	8 years ago
Eugen Rochko	412ea87306	Improve ActivityPub/OStatus compatibility (#4632) ... *Note: OStatus URIs are invalid for ActivityPub. But we have them for as long as we want to keep old OStatus-sourced content and as long as we remain OStatus-compatible.* - In Announce handling, if object URI is not a URL, fallback to object URL - Do not use specialized ThreadResolveWorker, rely on generalized handling - When serializing notes, if parent's URI is not a URL, use parent's URL	8 years ago
Eugen Rochko	10489b4e4a	If url attribute not present in Note, fallback to id attribute (#4629)	8 years ago
Eugen Rochko	ad892dbc0c	Add _:atomUri property for deduplicating OStatus/ActivityPub legacy records (#4593)	8 years ago
Eugen Rochko	dd7ef0dc41	Add ActivityPub inbox (#4216) ... * Add ActivityPub inbox * Handle ActivityPub deletes * Handle ActivityPub creates * Handle ActivityPub announces * Stubs for handling all activities that need to be handled * Add ActivityPub actor resolving * Handle conversation URI passing in ActivityPub * Handle content language in ActivityPub * Send accept header when fetching actor, handle JSON parse errors * Test for ActivityPub::FetchRemoteAccountService * Handle public key and icon/image when embedded/as array/as resolvable URI * Implement ActivityPub::FetchRemoteStatusService * Add stubs for more interactions * Undo activities implemented * Handle out of order activities * Hook up ActivityPub to ResolveRemoteAccountService, handle Update Account activities * Add fragment IDs to all transient activity serializers * Add tests and fixes * Add stubs for missing tests * Add more tests * Add more tests	8 years ago