You cannot select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

53 KiB

Encoding systems: ASCII, PETSCII, Unicode

Encoding systems

The process of converting from one type of information to another is called encoding. You can think of it as a similar process to Morse Code where longer and shorter beeps represent letters.

(Exercise: try to use the encoding system above to decode this: .... . .-.. .-.. ---- .-- ---- .-. -..)

Or the process of dialing a phone number, where pressing the keys generate a specific sound which tells the computers at the phone central who to connect you to.

Using the technique of encoding and decoding binary numbers can be made to represent everthing you see on the computer. There are standards, conventions which describe how something should be encoded and decoded so that one type of information can be translated in multiple places.

n.b. That encoding is not the same as encrypting! In the process of encoding the mapping between the code and the desired result are known and communicated. In encryption that mapping is purposely kept a secret.

Text encodings

As a computer can only work with numbers, it cannot process of letters text directly. In order to work with text, textual characters need to be translated into numbers and vice versa. This is done via the process of text encodings.

It might be your first reaction to think that this shouldn't be so difficult. We could represent the letter in binary code. An a encoded as a 0, b as a 1 and c as a 01 etc. And in fact this is more or less how text encodings work. However, at the time when computing was being developed different encodings emerged.

ASCII encoding

ASCII table

The dominant encoding at the time became ASCII (for American Standard Code for Information Interchange.), which was created on behalf of the U.S. Government in 1963 to allow for information interchange between their different computing systems.

The encoding uses a 7-bit system, which means that they could only store characters in 128 (2^7=128) numbers (0000 0000 until 0111 1111). The resulting encoding schema assigned to each of these 128 numbers:

  • all the letters in the English alphabet
  • numbers from 0-9
  • punctuation marks
  • and control characters

Thanks to the simplicity of the encoding it quickly became a standard for the American computing industry.

ASCII imperialism

Thanks to the power of the US Military and US corporations the American computing industry became the global computing industry. Computers that we use today are rooted in American networking history, and so is the ASCII standard. However, the reality is that ASCII can only represent 26 Latin letters in the English alphabet but computers are used all over the world, by people speaking different languages. They would often end up with American computers that could not represent their language in ASCII. Think for example of scripts like Greek, Cyrillic and Arabic or even Latin scripts that use accents such as the ü or ø. Altough 128 might sound like a lot of characters, it is not enough to represent all different languages.

Exercise - the limits of ASCII

Not all the characters fit within 128 or even 256 bits.

Try this for example:

In [1]:
! python2.7 -c "print 'ø'.encode('ascii')"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 0: ordinal not in range(128)

For a time people used the 8th bit, numbers 1000 0000 to 1111 1111 (or 128 to 256) to encode the specific parts of their own language. That way there was an overlap with ASCII but the own language could also be encoded.

A list of all the 128 ASCII characters, and their corresponding numbers can be seen in this table:

In [2]:
! man ascii
ASCII(7)                   Linux Programmer's Manual                  ASCII(7)

NAME
       ascii - ASCII character set encoded in octal, decimal, and hexadecimal

DESCRIPTION
       ASCII is the American Standard Code for Information Interchange.  It is
       a 7-bit code.  Many 8-bit codes (e.g., ISO  8859-1)  contain  ASCII  as
       their  lower  half.  The international counterpart of ASCII is known as
       ISO 646-IRV.

       The following table contains the 128 ASCII characters.

       C program '\X' escapes are noted.

       Oct   Dec   Hex   Char                        Oct   Dec   Hex   Char
       ────────────────────────────────────────────────────────────────────────
       000   0     00    NUL '\0' (null character)   100   64    40    @
       001   1     01    SOH (start of heading)      101   65    41    A
       002   2     02    STX (start of text)         102   66    42    B
       003   3     03    ETX (end of text)           103   67    43    C
       004   4     04    EOT (end of transmission)   104   68    44    D
       005   5     05    ENQ (enquiry)               105   69    45    E
       006   6     06    ACK (acknowledge)           106   70    46    F
       007   7     07    BEL '\a' (bell)             107   71    47    G
       010   8     08    BS  '\b' (backspace)        110   72    48    H
       011   9     09    HT  '\t' (horizontal tab)   111   73    49    I
       012   10    0A    LF  '\n' (new line)         112   74    4A    J
       013   11    0B    VT  '\v' (vertical tab)     113   75    4B    K
       014   12    0C    FF  '\f' (form feed)        114   76    4C    L
       015   13    0D    CR  '\r' (carriage ret)     115   77    4D    M
       016   14    0E    SO  (shift out)             116   78    4E    N
       017   15    0F    SI  (shift in)              117   79    4F    O
       020   16    10    DLE (data link escape)      120   80    50    P
       021   17    11    DC1 (device control 1)      121   81    51    Q
       022   18    12    DC2 (device control 2)      122   82    52    R
       023   19    13    DC3 (device control 3)      123   83    53    S
       024   20    14    DC4 (device control 4)      124   84    54    T
       025   21    15    NAK (negative ack.)         125   85    55    U
       026   22    16    SYN (synchronous idle)      126   86    56    V
       027   23    17    ETB (end of trans. blk)     127   87    57    W
       030   24    18    CAN (cancel)                130   88    58    X
       031   25    19    EM  (end of medium)         131   89    59    Y
       032   26    1A    SUB (substitute)            132   90    5A    Z
       033   27    1B    ESC (escape)                133   91    5B    [
       034   28    1C    FS  (file separator)        134   92    5C    \  '\\'
       035   29    1D    GS  (group separator)       135   93    5D    ]
       036   30    1E    RS  (record separator)      136   94    5E    ^
       037   31    1F    US  (unit separator)        137   95    5F    _
       040   32    20    SPACE                       140   96    60    `
       041   33    21    !                           141   97    61    a
       042   34    22    "                           142   98    62    b
       043   35    23    #                           143   99    63    c
       044   36    24    $                           144   100   64    d
       045   37    25    %                           145   101   65    e
       046   38    26    &                           146   102   66    f
       047   39    27    '                           147   103   67    g
       050   40    28    (                           150   104   68    h
       051   41    29    )                           151   105   69    i
       052   42    2A    *                           152   106   6A    j
       053   43    2B    +                           153   107   6B    k
       054   44    2C    ,                           154   108   6C    l
       055   45    2D    -                           155   109   6D    m

       056   46    2E    .                           156   110   6E    n
       057   47    2F    /                           157   111   6F    o
       060   48    30    0                           160   112   70    p
       061   49    31    1                           161   113   71    q
       062   50    32    2                           162   114   72    r
       063   51    33    3                           163   115   73    s
       064   52    34    4                           164   116   74    t
       065   53    35    5                           165   117   75    u
       066   54    36    6                           166   118   76    v
       067   55    37    7                           167   119   77    w
       070   56    38    8                           170   120   78    x
       071   57    39    9                           171   121   79    y
       072   58    3A    :                           172   122   7A    z
       073   59    3B    ;                           173   123   7B    {
       074   60    3C    <                           174   124   7C    |
       075   61    3D    =                           175   125   7D    }
       076   62    3E    >                           176   126   7E    ~
       077   63    3F    ?                           177   127   7F    DEL

   Tables
       For convenience, below are more compact tables in hex and decimal.

          2 3 4 5 6 7       30 40 50 60 70 80 90 100 110 120
        -------------      ---------------------------------
       0:   0 @ P ` p     0:    (  2  <  F  P  Z  d   n   x
       1: ! 1 A Q a q     1:    )  3  =  G  Q  [  e   o   y
       2: " 2 B R b r     2:    *  4  >  H  R  \  f   p   z
       3: # 3 C S c s     3: !  +  5  ?  I  S  ]  g   q   {
       4: $ 4 D T d t     4: "  ,  6  @  J  T  ^  h   r   |
       5: % 5 E U e u     5: #  -  7  A  K  U  _  i   s   }
       6: & 6 F V f v     6: $  .  8  B  L  V  `  j   t   ~
       7: ' 7 G W g w     7: %  /  9  C  M  W  a  k   u  DEL
       8: ( 8 H X h x     8: &  0  :  D  N  X  b  l   v
       9: ) 9 I Y i y     9: '  1  ;  E  O  Y  c  m   w
       A: * : J Z j z
       B: + ; K [ k {
       C: , < L \ l |
       D: - = M ] m }
       E: . > N ^ n ~
       F: / ? O _ o DEL

NOTES
   History
       An ascii manual page appeared in Version 7 of AT&T UNIX.

       On older terminals, the underscore code is displayed as a  left  arrow,
       called  backarrow, the caret is displayed as an up-arrow and the verti
       cal bar has a hole in the middle.

       Uppercase and lowercase characters differ by just one bit and the ASCII
       character  2  differs from the double quote by just one bit, too.  That
       made it much easier to encode characters mechanically or with a non-mi
       crocontroller-based  electronic  keyboard and that pairing was found on
       old teletypes.

       The ASCII standard was published by the United States of America  Stan
       dards Institute (USASI) in 1968.

SEE ALSO
       charsets(7),     iso_8859-1(7),     iso_8859-10(7),     iso_8859-11(7),
       iso_8859-13(7),   iso_8859-14(7),    iso_8859-15(7),    iso_8859-16(7),
       iso_8859-2(7),     iso_8859-3(7),     iso_8859-4(7),     iso_8859-5(7),
       iso_8859-6(7), iso_8859-7(7), iso_8859-8(7), iso_8859-9(7), utf-8(7)

COLOPHON
       This page is part of release 4.16 of the Linux  man-pages  project.   A
       description  of  the project, information about reporting bugs, and the
       latest    version    of    this    page,    can     be     found     at
       https://www.kernel.org/doc/man-pages/.

Linux                             2016-10-08                          ASCII(7)

Exercise - decode binary into ASCII

Use an ASCII code table and decode the following binary code:

0100 1000
0110 0101
0110 1100
0110 1100
0110 1111
0010 0000
0101 0111
0110 1111
0111 0010
0110 1100
0110 0100
0010 0001

ASCII flavours: PETSCII

Commodore 64 (1982)

The Commodore 64, also known as the C64 or the CBM 64, is an 8-bit home computer introduced in January 1982 by Commodore International (first shown at the Consumer Electronics Show, January 710, 1982, in Las Vegas). It has been listed in the Guinness World Records as the highest-selling single computer model of all time, with independent estimates placing the number sold between 12.5 and 17 million units.

Preceded by the Commodore VIC-20 and Commodore PET, the C64 took its name from its 64 kilobytes (65,536 bytes) of RAM. With support for multicolor sprites and a custom chip for waveform generation, the C64 could create superior visuals and audio compared to systems without such custom hardware.

https://en.wikipedia.org/wiki/Commodore_64

Part of the Commodore 64's success was its sale in regular retail stores instead of only electronics or computer hobbyist specialty stores. Commodore produced many of its parts in-house to control costs, including custom integrated circuit chips from MOS Technology. In the United States, it has been compared to the Ford Model T automobile for its role in bringing a new technology to middle-class households via creative and affordable mass-production.

Kahney, Leander (September 9, 2003). "Grandiose Price for a Modest PC". CondéNet, Inc. Archived from the original on September 14, 2008. Retrieved September 13, 2008.

PETSCII

The Commodore PET's lack of a programmable bitmap-mode for computer graphics, as well as it having no redefinable character set capability, may be one of the reasons PETSCII was developed; by creatively using the well-thought-out block graphics, a higher degree of sophistication in screen graphics is attainable than by using plain ASCII's letter/digit/punctuation characters. In addition to the relatively diverse set of geometrical shapes that can thus be produced, PETSCII allows for several grayscale levels by its provision of differently hatched checkerboard squares/half-squares. Finally, the reverse-video mode (see below) is used to complete the range of graphics characters, in that it provides mirrored half-square blocks.

https://en.wikipedia.org/wiki/PETSCII

Draw PETSCII art in the browser:

Use PETSCII as a font!

PETSCII bots!

Unicode universalism

'As electronic text was increasingly being exchanged online and between language areas, issues emerged when text encoded in one language was shared and read on systems assuming an encoding in another language. Unicode was a response to the incompatible text encoding standards that were proliferating.

When different encodings assign the same binary numbers to different characters, this results in illegible documents. The solution, partly made possible by increased computing capacity, was to strive for a single universal encoding which would encompass all writing systems'

(Roel Roscam Abbing, Peggy Pierrot, Femke Snelting (2016), Modifying the Universal)

You can experience this following the "break Yahoo" exercise below.

So in order to overcome the limitations of ASCII people created the Unicode Consortium to create a single universal character encoding:

'The Unicode standards are designed to normalise the encoding of characters, to efficiently manage the way they are stored, referred to and displayed in order to facilitate cross-platform, multilingual and international text exchange. The Unicode Standard is mammoth in size and covers well over 110,000 characters, of which [..] 1,000 are [..] emoji.'

(Roel Roscam Abbing, Peggy Pierrot, Femke Snelting (2016), Modifying the Universal)

In effect the Unicode Standard combined all the different national character encodings together into a single large ledger in order to try to represent all languages. It is divided in so called blocks, which are basically number tables that describe which number is connected to which character. The table starts counting at 0x0 and continues all the way up to 0x10FFFF.

The first block actually corresponds with ASCII: https://en.wikibooks.org/wiki/Unicode/Character_reference/0000-0FFF

It contains many different scripts for supporting large and smaller language groups, including for example Ethiopian and Cherokee: https://en.wikibooks.org/wiki/Unicode/Character_reference/1000-1FFF

However there are also blocks that describe Arrows and other symbols: https://en.wikibooks.org/wiki/Unicode/Character_reference/2000-2FFF

Emoji are also part of the unicode table.

From Unicode codepoint, to character, to glyph

Image representing a Unicode codepoint, a character and various glyphs

In the Unicode standard every text character has a representation as a number or codepoint. The standard defines what codepoint is connected to what character, but not what glyph should look like. That is left to individual font designers.

Next to the letters, there are many control characters that are used for example to display text from right to left and reverse or to join two separate characters together into one (as happens with Asian language scripts)

One of the curious things about Unicode is that there are many homographs (characters look identical, have different code points).

For example: Greek Ο, Latin O, and Cyrillic О are identical to the eye, but different to the computer.

You can tell when you take a closer look:

Ο = 0x39f for Greek
O = 0x4f for Latin О = 0x41e for Cyrillic

Try to copy the Greek or Cyrillic О and search for it in the document by pressing ctrl + f and entering it in the search bar. Then do the same with the Latin O.

Exercise - generating "Unicode-codepoint-art"

Here is a piece of python code that will fill try to print out all Unicode characters by counting from 0x0 to 0x10FFFF and converting that into a printable character:

In [3]:
! python -c 'exec """\nimport time\nwhile True:\n for i in range(0x0,0x10FFFF):\n  print unichr(i)*80\n  time.sleep(0.05)\n"""'








																																																																																






































































































                                                                                
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
################################################################################
$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&
''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((
))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))
********************************************************************************
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
--------------------------------------------------------------------------------
................................................................................
////////////////////////////////////////////////////////////////////////////////
00000000000000000000000000000000000000000000000000000000000000000000000000000000
11111111111111111111111111111111111111111111111111111111111111111111111111111111
22222222222222222222222222222222222222222222222222222222222222222222222222222222
33333333333333333333333333333333333333333333333333333333333333333333333333333333
44444444444444444444444444444444444444444444444444444444444444444444444444444444
55555555555555555555555555555555555555555555555555555555555555555555555555555555
66666666666666666666666666666666666666666666666666666666666666666666666666666666
77777777777777777777777777777777777777777777777777777777777777777777777777777777
88888888888888888888888888888888888888888888888888888888888888888888888888888888
99999999999999999999999999999999999999999999999999999999999999999999999999999999
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
================================================================================
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
????????????????????????????????????????????????????????????????????????????????
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC
DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD
EEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEE
FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF
GGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGG
HHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHH
IIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII
JJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJ
KKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKK
LLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLL
MMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMM
NNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNN
OOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOO
PPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPP
QQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQ
RRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRR
SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
TTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTT
UUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUU
VVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVV
WWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWW
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
YYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY
ZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZ
[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[
\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\
]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
________________________________________________________________________________
````````````````````````````````````````````````````````````````````````````````
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb
cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc
dddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddd
eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee
ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff
gggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggg
hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii
jjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjj
kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk
llllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllll
mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm
nnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnn
oooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo
pppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppp
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr
ssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss
tttttttttttttttttttttttttttttttttttttttttttttttttttttttttttttttttttttttttttttttt
^C
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "<string>", line 6, in <module>
KeyboardInterrupt

With this technique it is also possible to make animations:

In [4]:
! python -c 'exec """\nimport time\nwhile True:\n for i in range(127761,127768):\n  print unichr(i)*10\n  time.sleep(0.05)\n"""'
🌑🌑🌑🌑🌑🌑🌑🌑🌑🌑
🌒🌒🌒🌒🌒🌒🌒🌒🌒🌒
🌓🌓🌓🌓🌓🌓🌓🌓🌓🌓
🌔🌔🌔🌔🌔🌔🌔🌔🌔🌔
🌕🌕🌕🌕🌕🌕🌕🌕🌕🌕
🌖🌖🌖🌖🌖🌖🌖🌖🌖🌖
🌗🌗🌗🌗🌗🌗🌗🌗🌗🌗
🌑🌑🌑🌑🌑🌑🌑🌑🌑🌑
🌒🌒🌒🌒🌒🌒🌒🌒🌒🌒
🌓🌓🌓🌓🌓🌓🌓🌓🌓🌓
🌔🌔🌔🌔🌔🌔🌔🌔🌔🌔
🌕🌕🌕🌕🌕🌕🌕🌕🌕🌕
🌖🌖🌖🌖🌖🌖🌖🌖🌖🌖
🌗🌗🌗🌗🌗🌗🌗🌗🌗🌗
🌑🌑🌑🌑🌑🌑🌑🌑🌑🌑
🌒🌒🌒🌒🌒🌒🌒🌒🌒🌒
🌓🌓🌓🌓🌓🌓🌓🌓🌓🌓
🌔🌔🌔🌔🌔🌔🌔🌔🌔🌔
🌕🌕🌕🌕🌕🌕🌕🌕🌕🌕
🌖🌖🌖🌖🌖🌖🌖🌖🌖🌖
🌗🌗🌗🌗🌗🌗🌗🌗🌗🌗
🌑🌑🌑🌑🌑🌑🌑🌑🌑🌑
🌒🌒🌒🌒🌒🌒🌒🌒🌒🌒
🌓🌓🌓🌓🌓🌓🌓🌓🌓🌓
🌔🌔🌔🌔🌔🌔🌔🌔🌔🌔
🌕🌕🌕🌕🌕🌕🌕🌕🌕🌕
🌖🌖🌖🌖🌖🌖🌖🌖🌖🌖
🌗🌗🌗🌗🌗🌗🌗🌗🌗🌗
🌑🌑🌑🌑🌑🌑🌑🌑🌑🌑
🌒🌒🌒🌒🌒🌒🌒🌒🌒🌒
🌓🌓🌓🌓🌓🌓🌓🌓🌓🌓
🌕🌕🌕🌕🌕🌕🌕🌕🌕🌕
^C
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "<string>", line 6, in <module>
KeyboardInterrupt

Exercise - what does a japanese website look like in ASCII?

  1. Go to https://www.yahoo.co.jp/ in your browser.
  2. Save the page to your computer. You could use CMD+S or CTRL+S for this.
  3. Open the page in a text editor.
  4. Edit line 3 of the file: <meta http-equiv="content-type" content="text/html; charset=UTF-8"> and change the encoding of the document from UTF-8 into ASCII.
  5. Open the page.

You have now Mojibaked the page!

(Mojibake article on Wikipedia)

In [ ]: