Commit Graph

3 Commits

Author SHA1 Message Date
Christian Helmuth
a97b8043b5 utf8: non-character U+fffe as invalid codepoint
Unicode non-characters [1] are guaranteed to never be used for a
character. The formerly used U+fffd however is a valid character - the
replacement character [2] correctly displayed by Qt5 as <?>.

[1] https://en.wikipedia.org/wiki/Universal_Character_Set_characters#Non-characters
[2] https://en.wikipedia.org/wiki/Specials_(Unicode_block)#Replacement_character

Issue #3483
2019-08-28 14:22:32 +02:00
Emery Hemingway
22327b43ae Refactor terminal for intrinsic Unicode support
Refactor the graphical terminal server to internally represent
characters as 16-bit codepoints and handle the duplex terminal stream as
UTF-8.

- Make the Codepoint class printable to the Output interface
- Decode data received at the Terminal session from UTF-8 to a 16-bit
  character
- Pass 16-bit characters through terminal decoder and char-cell arrays
- Send Unicode through terminal session in a burst of UTF-8 bytes

Fix #3148
2019-02-19 11:08:17 +01:00
Norman Feske
d59d07b5e3 os: util/utf8.h for UTF-8 string handling
This patch adds a simple UTF-8 decoder at 'os/include/util/utf8.h'
along with a test at 'os/run/utf8.run'.

Fixes #2717, related to issue #2716
2018-03-27 13:44:27 +02:00