Commit Graph

14 Commits (5605b828e5149bd3a5161eb8d536ad51094afe64)

Author SHA1 Message Date
BSKY fccf83e1f2 Add noopener and/or noreferrer ()
Eugen Rochko 9ba40a6bfd
Remove HEAD request from fetching link previews ()
It is not really necessary and we need to reduce requests
ThibG e709b8da0d Ignore low-confidence CharlockHolmes guesses when parsing link cards ()
* Add failing test for windows-1251 link cards

* Ignore low-confidence CharlockHolmes guesses

Fixes 

* Fix no method error when charlock holmes cannot detect charset
Eugen Rochko cb5b5cb5f7
Slightly reduce RAM usage ()
* No need to re-require sidekiq plugins, they are required via Gemfile

* Add derailed_benchmarks tool, no need to require TTY gems in Gemfile

* Replace ruby-oembed with FetchOEmbedService

Reduce startup by 45382 allocated objects

* Remove preloaded JSON-LD in favour of caching HTTP responses

Reduce boot RAM by about 6 MiB

* Fix tests

* Fix test suite by stubbing out JSON-LD contexts
Daniel King 6ef3874b2e Fix URLs incorrectly having trailing hyphen removed ()
In cases where a URL has a trailing hyphen the FetchLinkCardService incorrectly removes the hyphen when it is parsed

The hyphen is not a reserved character in the URI spec https://tools.ietf.org/html/rfc3986#section-2.2
ふぁぼ原 3816943e6b Enable to recognize most kinds of characters as URL paths ()
Eugen Rochko e7adbf572a Switch to static URIs, new URI format in both protocols for new statuses ()
* Decouple Status#local? from uri being nil

* Replace on-the-fly URI generation with stored URIs

- Generate URI in after_save hook for local statuses
- Use static value in TagManager when available, fallback to tag format
- Make TagManager use ActivityPub::TagManager to understand new format
- Adjust tests

* Use other heuristic for locality of old statuses, do not perform long query

* Exclude tombstone stream entries from Atom feed

* Prevent nil statuses from landing in Pubsubhubbub::DistributionWorker

* Fix URI not being saved ()

* Add more specs for Status

* Save generated uri immediately

and also fix method order to minimize diff.

* Fix alternate HTML URL in Atom

* Fix tests

* Remove not-null constraint from statuses migration to speed it up
Eugen Rochko 7dc5035031 Make PreviewCard records reuseable between statuses ()
* Make PreviewCard records reuseable between statuses

**Warning!** Migration truncates preview_cards tablec

* Allow a wider thumbnail for link preview, display it in horizontal layout ()

* Delete preview cards files before truncating

* Rename old table instead of truncating it

* Add mastodon:maintenance:remove_deprecated_preview_cards

* Ignore deprecated_preview_cards in schema definition

* Fix null behaviour
nullkal 007ab330e6 Use charlock_holmes instead of nkf at FetchLinkCardService ()
* Specs for language detection

* Use CharlockHolmes instead of NKF

* Correct mistakes

* Correct style

* Set hint_enc instead of falling back and strip_tags

* Improve specs

* Add dependencies
abcang 5e6acf9601 Fix Nokogiri::HTML at FetchLinkCardService ()
Eugen Rochko 1efda1c453 Fix - Resolve preview cards for remote statuses as well as local ones ()
Eugen Rochko a231f915a0 Fix - Send HEAD request ahead of GET when fetching URL previews ()
Eugen Rochko 88725d6ce8 OEmbed support for PreviewCard ()
* OEmbed support for PreviewCard

* Improve ProviderDiscovery code failure treatment

* Do not crawl links if there is a content warning, since those
don't display a link card anyway

* Reset db schema

* Fresh migrate

* Fix rubocop style issues
Fix  - return existing access token when applicable instead of creating new

* Fix test

* Extract http client to helper

* Improve oembed controller
Eugen 17c591ffba Punycode URI normalization ()
* Fix  - Whenever about to send a HTTP request, normalize the URI

* Add test for IDN request in FetchLinkCardService

* Perform IDN normalization on domains before they are stored in the DB