Allow Unicode letters in story text overlays; block invisible chars

The text-overlay validator on POST /api/v1.2/stories/publish used a
character whitelist of `[a-zA-Z0-9\s\p{P}\p{S}]`, which rejected any
overlay containing Unicode letters — German umlauts (ä, ö, ü, ß), accents
(é, à, ñ), Cyrillic, Arabic, CJK — with HTTP 500 and the message
"Text overlays contain unsupported characters." The iOS app surfaces this
as a generic "contact admin" error.

Replace the whitelist with `[\p{L}\p{M}\p{N}\p{Zs}\p{P}\p{S}]` and extract
it to a class constant so it can be unit-tested.

Accepts:
- Unicode letters (\p{L}) and combining marks (\p{M}) — covers umlauts,
  accents in both NFC and NFD form, non-Latin scripts.
- Digits (\p{N}), space separators (\p{Zs}), punctuation (\p{P}).
- Symbols (\p{S}) — single-codepoint emoji, regional-indicator country
  flags (🇩🇪), currency, math.

Rejects, by omission:
- Control chars (\p{Cc}) — NUL, tab, newline. A single-line story
  overlay shouldn't carry these, and they're common injection vectors.
- Format chars (\p{Cf}) — zero-width space (U+200B), zero-width
  non-joiner/joiner (U+200C/U+200D), word joiner (U+2060), BOM
  (U+FEFF), bidi overrides (U+202A–U+202E), directional isolates
  (U+2066–U+2069), Arabic letter mark (U+061C), Mongolian vowel
  separator (U+180E), tag characters (U+E0000–U+E007F). These are
  invisible characters used for spoofing, homoglyph attacks and
  bidi-override exploits.
- Private use (\p{Co}), unassigned (\p{Cn}), surrogates (\p{Cs}).

Uses `\p{Zs}` rather than `\s` because Perl-compatible `\s` also matches
some legacy format characters such as U+180E even when those are
classified as `\p{Cf}` in the current Unicode database.

Known tradeoff: ZWJ-composed emoji sequences (👨‍👩‍👧, 🏳️‍🌈, profession and
skin-tone variants) are rejected because U+200D ZERO WIDTH JOINER is
`\p{Cf}`. This is deliberate — keeping zero-width characters out of
overlay text is more valuable than supporting composite emoji here. The
single-codepoint emoji set and country flags via regional indicators
still work.

Adds tests/Unit/Stories/StoryTextOverlayPatternTest.php covering 33
boundary cases.
pull/6612/head
vinzgreg 2 months ago
parent ef3fdeeca3
commit e34dbc6c94

@ -35,6 +35,28 @@ class StoryApiV1Controller extends Controller
const RECENT_TTL = 300;
/**
* Allowed characters in story text overlays.
*
* Accepts: Unicode letters (\p{L}), combining marks (\p{M}), digits
* (\p{N}), space separators (\p{Zs}, including ASCII space and NBSP),
* punctuation (\p{P}), and symbols (\p{S}, which includes single-
* codepoint emoji and regional-indicator flags).
*
* Rejects, by omission: control chars (\p{Cc} — incl. tab, newline,
* NUL), format chars (\p{Cf} — zero-width space/joiner, BOM, bidi
* overrides, tag chars, Mongolian vowel separator), private use
* (\p{Co}), unassigned (\p{Cn}), surrogates (\p{Cs}). Uses \p{Zs}
* rather than \s because Perl-compat \s also matches some legacy
* format characters like U+180E. This blocks the common invisible-
* character and bidi-spoofing attack vectors.
*
* Tradeoff: ZWJ-composed emoji sequences (e.g. 👨‍👩‍👧, 🏳️‍🌈) are rejected
* because U+200D ZERO WIDTH JOINER is \p{Cf}. Single-codepoint emoji
* and regional-indicator country flags still work.
*/
const TEXT_OVERLAY_PATTERN = '/^[\p{L}\p{M}\p{N}\p{Zs}\p{P}\p{S}]*$/u';
public function carousel(Request $request)
{
abort_if(! (bool) config_cache('instance.stories.enabled') || ! $request->user(), 404);
@ -444,7 +466,7 @@ class StoryApiV1Controller extends Controller
switch ($type) {
case 'text':
if (! preg_match('/^[a-zA-Z0-9\s\p{P}\p{S}]*$/u', $content)) {
if (! preg_match(self::TEXT_OVERLAY_PATTERN, $content)) {
throw ValidationException::withMessages([
"overlays.{$index}.content" => 'Text overlays contain unsupported characters.',
]);

@ -0,0 +1,85 @@
<?php
namespace Tests\Unit\Stories;
use App\Http\Controllers\Stories\StoryApiV1Controller;
use PHPUnit\Framework\Attributes\DataProvider;
use PHPUnit\Framework\Attributes\Test;
use PHPUnit\Framework\TestCase;
/**
* Guards the text-overlay character whitelist used by
* StoryApiV1Controller::publishNext(). Accepts a wide range of
* legitimate Unicode (umlauts, accents, CJK, single-codepoint
* emoji, regional-indicator flags) while rejecting invisible /
* bidi-spoofing characters.
*/
class StoryTextOverlayPatternTest extends TestCase
{
public static function acceptedProvider(): array
{
return [
'ascii' => ['Hello World 123'],
'german umlaut' => ['Hallöchen Müller'],
'accent nfc' => ['café'],
'accent nfd' => ["cafe\u{0301}"],
'cyrillic' => ['Привет'],
'arabic' => ['مرحبا'],
'cjk' => ['你好世界'],
'single emoji' => ['Hi 😀'],
'heart vs16' => ["I\u{2764}\u{FE0F}NY"],
'country flag' => ['Vacation 🇩🇪'],
'currency symbols' => ['Price: 100 € or $100'],
'math symbols' => ['x²+y²=z²'],
'punctuation' => ['Hello, world! (test) — "quoted"'],
'empty string' => [''],
];
}
public static function rejectedProvider(): array
{
return [
'zero-width space' => ["hello\u{200B}admin"],
'zero-width nojoin' => ["a\u{200C}b"],
'zero-width joiner' => ["a\u{200D}b"],
'word joiner' => ["ab\u{2060}cd"],
'bom / zwnbsp' => ["\u{FEFF}leading"],
'bidi rlo' => ["innocent\u{202E}cixot"],
'bidi lro' => ["\u{202D}override"],
'ltr isolate' => ["abc\u{2066}xyz\u{2069}"],
'rtl isolate' => ["abc\u{2067}xyz\u{2069}"],
'arabic letter mark' => ["a\u{061C}b"],
'mongolian vowel sep'=> ["a\u{180E}b"],
'private use area' => ["abc\u{E000}"],
'tag character' => ["abc\u{E0041}"],
'nul byte' => ["abc\x00def"],
'tab' => ["a\tb"],
'newline' => ["a\nb"],
'carriage return' => ["a\rb"],
'line separator' => ["a\u{2028}b"],
'paragraph separator'=> ["a\u{2029}b"],
];
}
#[Test]
#[DataProvider('acceptedProvider')]
public function pattern_accepts_safe_input(string $input): void
{
$this->assertSame(
1,
preg_match(StoryApiV1Controller::TEXT_OVERLAY_PATTERN, $input),
'Expected pattern to accept input: '.bin2hex($input)
);
}
#[Test]
#[DataProvider('rejectedProvider')]
public function pattern_rejects_invisible_or_bidi_input(string $input): void
{
$this->assertSame(
0,
preg_match(StoryApiV1Controller::TEXT_OVERLAY_PATTERN, $input),
'Expected pattern to reject input: '.bin2hex($input)
);
}
}
Loading…
Cancel
Save