Hi all,

echo -n 'Merry Christmas and a Happy New Year' | base16 -l 0
4d65727279204368726973746d617320616e642061204861707079204e65772059656172

Now we encrypt this message with fpe, a 256 bit key and a 64 bit nonce:

ca94a7a70929a2eb44faf7a9195541be16a97ab17e0a40cd95c81978f14fcdf6b76892ae

After that we convert the hex string to decimal

393547191367737629111313439284388256222575714261460221423875982235587707334230765245102

and finally group the string into 5 digits groups, to have an OTP
look a like message. :-)

39354 71913 67737 62911 13134 39284 38825 62225 75714 26146
02214 23875 98223 55877 07334 23076 52451 02

This is just for fun, but shows also for what hex2dec and vice versa
converters can be used.

Merry Christmas
Stefan
--
----Ed25519 Signature----
5fe71d0255bbc7f968b15ff1f4d5897e2d3211bd98687f1ed1d048f2616a08bd
6227903e8058f9c334684aecf602cdd7f0ce2e2a8d6eca94f530a9d9f33ab70d

Stefan Claas <stefan.claas@iris.to> wrote:
> Hi all,
>
> echo -n 'Merry Christmas and a Happy New Year' | base16 -l 0
> 4d65727279204368726973746d617320616e642061204861707079204e65772059656172
>
> Now we encrypt this message with fpe, a 256 bit key and a 64 bit nonce:
>
> ca94a7a70929a2eb44faf7a9195541be16a97ab17e0a40cd95c81978f14fcdf6b76892ae
>
> After that we convert the hex string to decimal
>
> 393547191367737629111313439284388256222575714261460221423875982235587707334230765245102
>
> and finally group the string into 5 digits groups, to have an OTP
> look a like message. :-)
>
> 39354 71913 67737 62911 13134 39284 38825 62225 75714 26146
> 02214 23875 98223 55877 07334 23076 52451 02
>
> This is just for fun, but shows also for what hex2dec and vice versa
> converters can be used.

Note that conversion to decimal is not required for grouping into
groups of five. The hex message can just as easily be grouped into
five digit groupings:

ca94a 7a709 29a2e b44fa f7a91 95541 be16a 97ab1 7e0a4 0cd95
c8197 8f14f cdf6b 76892 ae

Alternately, using the Ascii85 scheme
(https://en.wikipedia.org/wiki/Ascii85) you can get a smaller
"message":

$ rlwrap tclsh
% set binary [binary decode hex ca94a7a70929a2eb44faf7a9195541be16a97ab17e0a40cd95c81978f14fcdf6b76892ae] ; list
% package require ascii85
1.0
% ascii85::encode -maxlen 0 $binary
b*8g%#q;V77/HYW)-&u:(9)VbIL8?bQ,jhfnP@F'[qLr-

Which can itself be "grouped" if one likes:

b*8g% #q;V7 7/HYW )-&u: (9)Vb IL8?b Q,jhf nP@F' [qLr-

Of course, having a coms channel that passes binary data would avoid
needing to 'encode' the mesage as any of the "text/ascii" formats.

Rich wrote:

> Stefan Claas <stefan.claas@iris.to> wrote:

> > This is just for fun, but shows also for what hex2dec and vice versa
> > converters can be used.
>
> Note that conversion to decimal is not required for grouping into
> groups of five. The hex message can just as easily be grouped into
> five digit groupings:
>
> ca94a 7a709 29a2e b44fa f7a91 95541 be16a 97ab1 7e0a4 0cd95
> c8197 8f14f cdf6b 76892 ae

Yes, but in classic OTP messages I have never seen hex usage.
> Alternately, using the Ascii85 scheme
> (https://en.wikipedia.org/wiki/Ascii85) you can get a smaller
> "message":
>
> $ rlwrap tclsh
> % set binary [binary decode hex
> ca94a7a70929a2eb44faf7a9195541be16a97ab17e0a40cd95c81978f14fcdf6b76892ae]
> ; list % package require ascii85 1.0
> % ascii85::encode -maxlen 0 $binary
> b*8g%#q;V77/HYW)-&u:(9)VbIL8?bQ,jhfnP@F'[qLr-
>
> Which can itself be "grouped" if one likes:
>
> b*8g% #q;V7 7/HYW )-&u: (9)Vb IL8?b Q,jhf nP@F' [qLr-

Or one uses base91, which is a bit more smaller.

> Of course, having a coms channel that passes binary data would avoid
> needing to 'encode' the mesage as any of the "text/ascii" formats.

Yes. Nice tools for that, like croc, magic-wormhole, OnionShare etc.
are available on GitHub, for example.

Regards
Stefan
--
----Ed25519 Signature----
0afb5db0e17c2d6b8a8734739dd0e48d4a53f4e8246639c156110f3eada9b621
38b4fa8fb421ba9aaffd8f46803e4b714134f257fa1c35233e504ec8a43b9e06

Stefan Claas in sci.crypt:

>echo -n 'Merry Christmas and a Happy New Year' | base16 -l 0
>4d65727279204368726973746d617320616e642061204861707079204e65772059656172
>
>Now we encrypt this message with fpe, a 256 bit key and a 64 bit nonce:
>
>ca94a7a70929a2eb44faf7a9195541be16a97ab17e0a40cd95c81978f14fcdf6b76892ae
>
>After that we convert the hex string to decimal
>
>393547191367737629111313439284388256222575714261460221423875982235587707334230765245102

If the FPE ciphertext (accidentally) begins with "00" then there
is a problem to decode the decimal value back to hex, I think:

| user15@o15:/tmp$ BC_LINE_LENGTH=0 bc -q
| ibase=16
| 0094A7A70929A2EB44FAF7A9195541BE16A97AB17E0A40CD95C81978F14FCDF6B76892AE
| 1128075138140356785582134106037437080809532772928038131009875468329041722778648613550

| user15@o15:/tmp$ BC_LINE_LENGTH=0 bc -q
| obase=16
| 1128075138140356785582134106037437080809532772928038131009875468329041722778648613550
| 94A7A70929A2EB44FAF7A9195541BE16A97AB17E0A40CD95C81978F14FCDF6B76892AE

>Merry Christmas

Thanks. Same to you.

Marcel

Marcel Logen wrote:

> Stefan Claas in sci.crypt:
>
> >echo -n 'Merry Christmas and a Happy New Year' | base16 -l 0
> >4d65727279204368726973746d617320616e642061204861707079204e65772059656172
> >
> >Now we encrypt this message with fpe, a 256 bit key and a 64 bit
> >nonce:
> >
> >ca94a7a70929a2eb44faf7a9195541be16a97ab17e0a40cd95c81978f14fcdf6b76892ae
> >
> >After that we convert the hex string to decimal
> >
> >393547191367737629111313439284388256222575714261460221423875982235587707334230765245102
>
> If the FPE ciphertext (accidentally) begins with "00" then there
> is a problem to decode the decimal value back to hex, I think:
>
> | user15@o15:/tmp$ BC_LINE_LENGTH=0 bc -q
> | ibase=16
> |
> 0094A7A70929A2EB44FAF7A9195541BE16A97AB17E0A40CD95C81978F14FCDF6B76892AE
> |
> 1128075138140356785582134106037437080809532772928038131009875468329041722778648613550
>
> | user15@o15:/tmp$ BC_LINE_LENGTH=0 bc -q
> | obase=16
> |
> 1128075138140356785582134106037437080809532772928038131009875468329041722778648613550
> |
> 94A7A70929A2EB44FAF7A9195541BE16A97AB17E0A40CD95C81978F14FCDF6B76892AE

Mmmh, if you prepend the zeros to my message, you get the correct
decimal value. Your message is IMHO completely different, because
yours begins with (00)94 and not (00)ca, or am I wrong?

And should this happen, one could choose a new nonce, for the message,
I would say.

Regards
Stefan
--
----Ed25519 Signature----
3f5f81c970743d4d8c4928119522734140cd51ea7866370ef28bf1b71101875e
2c9be22e8048d7b488e877df7881ece65210628ca6f2ad26bdd29f80e5eefa04

Stefan Claas in sci.crypt:

>Marcel Logen wrote:

[...]

>> If the FPE ciphertext (accidentally) begins with "00" then there
>> is a problem to decode the decimal value back to hex, I think:
[...]

>Mmmh, if you prepend the zeros to my message, you get the correct
>decimal value. Your message is IMHO completely different, because
>yours begins with (00)94 and not (00)ca, or am I wrong?

This was simply a (constructed) example for clarification
of the problem.

>And should this happen, one could choose a new nonce, for the message,
>I would say.

This could be a solution, yes. However, it should be detected
and performed automatically by the FPE encryption routine.

Marcel

Stefan Claas <stefan.claas@iris.to> wrote:
> Rich wrote:
>
>> Stefan Claas <stefan.claas@iris.to> wrote:
>
>> > This is just for fun, but shows also for what hex2dec and vice versa
>> > converters can be used.
>>
>> Note that conversion to decimal is not required for grouping into
>> groups of five. The hex message can just as easily be grouped into
>> five digit groupings:
>>
>> ca94a 7a709 29a2e b44fa f7a91 95541 be16a 97ab1 7e0a4 0cd95
>> c8197 8f14f cdf6b 76892 ae
>
> Yes, but in classic OTP messages I have never seen hex usage.

Most likely due to 'classic OTP' being done by hand, using actual paper
pads of random numbers (that's where the "pad" part of the OTP name
derives from, an actual "note pad" style paper device:
(https://en.wikipedia.org/wiki/File:Legal_pad_and_pencil.jpg).

If performing the encrypt/decrypt by hand, it is likely easier to have
the pad be all decimal and leverage human base10 numbers vs.
additionally training each and every spy the meaning of hex values (or
other bases).

>> Alternately, using the Ascii85 scheme
>> (https://en.wikipedia.org/wiki/Ascii85) you can get a smaller
>> "message":
>>
>> $ rlwrap tclsh
>> % set binary [binary decode hex
>> ca94a7a70929a2eb44faf7a9195541be16a97ab17e0a40cd95c81978f14fcdf6b76892ae]
>> ; list % package require ascii85 1.0
>> % ascii85::encode -maxlen 0 $binary
>> b*8g%#q;V77/HYW)-&u:(9)VbIL8?bQ,jhfnP@F'[qLr-
>>
>> Which can itself be "grouped" if one likes:
>>
>> b*8g% #q;V7 7/HYW )-&u: (9)Vb IL8?b Q,jhf nP@F' [qLr-
>
> Or one uses base91, which is a bit more smaller.

Indeed, I was not aware that someone had one-upped ascii85. Base91 is
approaching about as tight as a scheme can get and still use only the
printable 7-bit ASCII code points.

On 2023-12-24 23:12, Marcel Logen wrote:
> Stefan Claas in sci.crypt:
>
>> Marcel Logen wrote:
>
> [...]
>
>>> If the FPE ciphertext (accidentally) begins with "00" then there
>>> is a problem to decode the decimal value back to hex, I think:
> [...]
>
>> Mmmh, if you prepend the zeros to my message, you get the correct
>> decimal value. Your message is IMHO completely different, because
>> yours begins with (00)94 and not (00)ca, or am I wrong?
>
> This was simply a (constructed) example for clarification
> of the problem.
>
>> And should this happen, one could choose a new nonce, for the message,
>> I would say.
>
> This could be a solution, yes. However, it should be detected
> and performed automatically by the FPE encryption routine.
>

Or just define the scheme to include enough leading zeroes to represent
all messages of the same number of bits, just like paper-based OTP ends
up with 0 as the first digit of about 10% of the messages (because the
secret digits are completely random and unrelated to the message).

With the command line tools this can be done as follows:

1. note the length of the actual hex string

2. Create a test hex string of the same length but all the hexits are F

3. Convert that test hex string to decimal.

4. note the length of the test decimal string

5. Output enough leading zeroes to make the actual decimal string as
long as the test decimal string.

6. To avoid creating side channel attacks, do steps 1 to 4 before
actually encrypting any message, based on how long the encrypted form of
messages would be. This takes a bit of arithmetic skill.

Enjoy

Jakob
--
Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com
Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10
This public discussion message is non-binding and may contain errors.
WiseMo - Remote Service Management for PCs, Phones and Embedded

Jakob Bohm in sci.crypt:

>On 2023-12-24 23:12, Marcel Logen wrote:
>>Stefan Claas in sci.crypt:
>>>Marcel Logen wrote:

>>>>If the FPE ciphertext (accidentally) begins with "00" then there
>>>>is a problem to decode the decimal value back to hex, I think:
>>[...]
>>
>>>Mmmh, if you prepend the zeros to my message, you get the correct
>>>decimal value. Your message is IMHO completely different, because
>>>yours begins with (00)94 and not (00)ca, or am I wrong?
>>
>>This was simply a (constructed) example for clarification
>>of the problem.
>>
>>>And should this happen, one could choose a new nonce, for the message,
>>>I would say.
>>
>>This could be a solution, yes. However, it should be detected
>>and performed automatically by the FPE encryption routine.
>
>Or just define the scheme to include enough leading zeroes to represent
>all messages of the same number of bits, just like paper-based OTP ends
>up with 0 as the first digit of about 10% of the messages (because the
>secret digits are completely random and unrelated to the message).
>
>With the command line tools this can be done as follows:
>
>1. note the length of the actual hex string
>
>2. Create a test hex string of the same length but all the hexits are F
>
>3. Convert that test hex string to decimal.
>
>4. note the length of the test decimal string
>
>5. Output enough leading zeroes to make the actual decimal string as
>long as the test decimal string.

Does this actually solve the problem (hex string begins with
"0" and this zero has to be restored by converting decimal to
hex)? ... pondering ...

>6. To avoid creating side channel attacks, do steps 1 to 4 before
>actually encrypting any message, based on how long the encrypted form of
>messages would be. This takes a bit of arithmetic skill.

Here my pre-computation of 1. to 4. (with GNU bash and GNU bc):

| user15@o15:/tmp$ cat script.bash
| #!/usr/bin/bash
| for i in {2..100..2}
| do var='' &&
| echo -n "$i " &&
| for j in $(seq "$i")
| do var="$var"'F'
| done &&
| echo "$var" &&
| echo -n "$i " &&
| xar=$( echo "ibase=16; ${var}" | BC_LINE_LENGTH=0 bc ) &&
| echo "$xar" && echo -n "$i " &&
| echo "$(( $( echo $xar | wc -c ) - 1 ))" &&
| echo
| done

| user15@o15:/tmp$ ./script.bash
| 2 FF
| 2 255
| 2 3
|
| 4 FFFF
| 4 65535
| 4 5
|
| 6 FFFFFF
| 6 16777215
| 6 8
|
| 8 FFFFFFFF
| 8 4294967295
| 8 10
|
| 10 FFFFFFFFFF
| 10 1099511627775
| 10 13
|
| 12 FFFFFFFFFFFF
| 12 281474976710655
| 12 15
|
[...]
| 96 FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF
| 96 39402006196394479212279040100143613805079739270465446667948293404245721771497210611414266254884915640806627990306815
| 96 116
|
| 98 FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF
| 98 10086913586276986678343434265636765134100413253239154346994763111486904773503285916522052161250538404046496765518544895
| 98 119
|
| 100 FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF
| 100 2582249878086908589655919172003011874329705792829223512830659356540647622016841194629645353280137831435903171972747493375
| 100 121
|

Marcel

Rich wrote:

> Stefan Claas <stefan.claas@iris.to> wrote:

> > Or one uses base91, which is a bit more smaller.
>
> Indeed, I was not aware that someone had one-upped ascii85. Base91
> is approaching about as tight as a scheme can get and still use only
> the printable 7-bit ASCII code points.

I just figured out that one can use base91 with base26 to encode small
binary blobs for the Diana Cryptosystem. :-)

Regards
Stefan
--
----Ed25519 Signature----
1dd63c208285b01a006798b8366d8089588d7c7a622c2afbb8d8d975ea432c67
8cb6771b9f1931f1e7b87bcedfa00a0887d2f513062f42b769af4c827a35b706

Stefan Claas wrote:

> I just figured out that one can use base91 with base26 to encode small
> binary blobs for the Diana Cryptosystem. :-)

And now I have a base26 Encoder/Decoder, which handles Unicode nicely,
according to a small test.

VCGBPEBELEAEGBMEAEXDGBLEVDBEUBVDKEREIEMEGBVDHEFEFENEGEBEMEREGBTD
GBAETDIEIEREGBGEXDPEGBREXDTDKEGBYBWBYBACHB

https://github.com/stefanclaas/base26

Regards
Stefan
--
----Ed25519 Signature----
431ecb0e14c13465ce7e9d09af4eefd986d9ad81165ba2d8629873fcd714dea6
f4cc2e79347a63a8b8c9347536b4e513d8bd9999637eadf04cfb8874650dda0a

Stefan Claas <stefan.claas@iris.to> wrote:
> Rich wrote:
>
>> Stefan Claas <stefan.claas@iris.to> wrote:
>
>> > Or one uses base91, which is a bit more smaller.
>>
>> Indeed, I was not aware that someone had one-upped ascii85. Base91
>> is approaching about as tight as a scheme can get and still use only
>> the printable 7-bit ASCII code points.
>
> I just figured out that one can use base91 with base26 to encode small
> binary blobs for the Diana Cryptosystem. :-)

But, why double up. If base26 already encodes binary, there is no
advantage to first base91 encoding it. All you achieve is further
increasing the size.

Rich wrote:

> Stefan Claas <stefan.claas@iris.to> wrote:

> > I just figured out that one can use base91 with base26 to encode
> > small binary blobs for the Diana Cryptosystem. :-)
>
> But, why double up. If base26 already encodes binary, there is no
> advantage to first base91 encoding it. All you achieve is further
> increasing the size.

As understood, base26 is not meant to handle binary and the various
implementations, I have seen, work all differently. But I could be
wrong of course.

Regards
Stefan
--
----Ed25519 Signature----
618f9b6a513c44dbbe41c4bfc81f74379b54b3b118f4da88082b1a465c764146
10fb57a41eb214d15c19ae1fb954ca9ff1fbadad16c8f0733b5641aaae848509

Stefan Claas <stefan@mailchuck.com> wrote:
> Rich wrote:
>
>> Stefan Claas <stefan.claas@iris.to> wrote:
>
>> > I just figured out that one can use base91 with base26 to encode
>> > small binary blobs for the Diana Cryptosystem. :-)
>>
>> But, why double up. If base26 already encodes binary, there is no
>> advantage to first base91 encoding it. All you achieve is further
>> increasing the size.
>
> As understood, base26 is not meant to handle binary and the various
> implementations, I have seen, work all differently. But I could be
> wrong of course.

All the other "baseXX" text encoding schemes are meant for encoding
binary data into a textual form that can pass through non-binary clean
communications methods. It would be unusual if a scheme named "base26"
was somehow different and unable to encode binary data into text. With
that said, a quick search shows that there is no explicit "base26"
encoding scheme, so it may just be that no one defined one yet to fit
with the others.

Although unless one is restricted on only the 26 English letters, using
one of the more compact schemes (base64, base85) reduces bloat. A
'base26' encoding could encode a max of four bits per letter, which
makes it the same 'expansion' factor as hex encoding, just using
letters instead of numerals and letters.

Rich wrote:

> Although unless one is restricted on only the 26 English letters,
> using one of the more compact schemes (base64, base85) reduces bloat.
> A 'base26' encoding could encode a max of four bits per letter,
> which makes it the same 'expansion' factor as hex encoding, just
> using letters instead of numerals and letters.

The idea behind my base26 Encoder/Decoder was to use it with, for
example, the Diana Cryptosystem, without a Signal Book, or Code Book.

Thus allowing people, speaking foreign languages, to use Diana as well.

It makes the messages much longer, of course, but until I create a
Signal Book, for my city, it will be a good replacement.

Example plain text, in German:

"Treffen heute Abend, um 20:00 Uhr am großen See, zur Sylvester Party."

Clear text for Diana (fiver-groups):

TREFF ENHEU TEABE DUM [Code group for time] AMGRO SSENS EEZUR SYLVE
STERP ARTY

Same message clearcode, base26:

GDKEX DYDYD XDGEG BAEXD NEMEX DGBNC UDXDG EWDSB GBNEF EGBYB
WBGCW BWBGB HDAEK EGBTD FEGBZ DKEHE NHDGX DGEGB FDXDX DSBGB
SENEK EGBFD REEEO EXDLE MEXDK EGBCD TDKEM EREUB

So, with an offline Computer and base26, one can use Diana
with any language.

P.S. same goes for my hex/dec Encoder/Decoder to use any
language with OTPs, consiting of digits.

Regards
Stefan
--
----Ed25519 Signature----
cf2322dbca106ff4a48b0c1d1f07ce634cf9582ff70d4c0b4e1a131063e7bb4e
69cbef600171e2fd20e4543e56e3be7d151912a1e0ac74480730fbedfffa9e05

Stefan Claas <stefan@mailchuck.com> wrote:
> Rich wrote:
>
>> Although unless one is restricted on only the 26 English letters,
>> using one of the more compact schemes (base64, base85) reduces bloat.
>> A 'base26' encoding could encode a max of four bits per letter,
>> which makes it the same 'expansion' factor as hex encoding, just
>> using letters instead of numerals and letters.
>
> The idea behind my base26 Encoder/Decoder was to use it with, for
> example, the Diana Cryptosystem, without a Signal Book, or Code Book.
>
> Thus allowing people, speaking foreign languages, to use Diana as well.
>
> It makes the messages much longer, of course, but until I create a
> Signal Book, for my city, it will be a good replacement.
>
> Example plain text, in German:
>
> "Treffen heute Abend, um 20:00 Uhr am großen See, zur Sylvester Party."
>
> Clear text for Diana (fiver-groups):
>
> TREFF ENHEU TEABE DUM [Code group for time] AMGRO SSENS EEZUR SYLVE
> STERP ARTY
>
> Same message clearcode, base26:
>
> GDKEX DYDYD XDGEG BAEXD NEMEX DGBNC UDXDG EWDSB GBNEF EGBYB
> WBGCW BWBGB HDAEK EGBTD FEGBZ DKEHE NHDGX DGEGB FDXDX DSBGB
> SENEK EGBFD REEEO EXDLE MEXDK EGBCD TDKEM EREUB
>
> So, with an offline Computer and base26, one can use Diana
> with any language.

In which case you don't need to baseXX and then base26. Just define
your own base26 that is a binary to text encoder. It is effectively
just a "hex" encoder with each of zero through nine replaced by an
English alphabet letter. So you could just "hex" encode using an
existing hex encoder library (saving yourself the trouble of writing
'bit twiddling' code), and then just character map zero through nine
onto letters. You could do the map in at least two simple ways:

Input: 0123456789ABCDEF
Output: GHIJKLMNOPABCDEF

Input: 0123456789ABCDEF
Output: ABCDEFGHIJKLMNOP

Rich wrote:

> Stefan Claas <stefan@mailchuck.com> wrote:

> > So, with an offline Computer and base26, one can use Diana
> > with any language.
>
> In which case you don't need to baseXX and then base26. [...]

Correct! Now my base26 Encoder/Decoder handles Unicode.

Regards
Stefan
--
----Ed25519 Signature----
689c57dc8607b3408f21d9ea39ae7dd083c2fdbe60bbdaeda3f57ce783593a71
1e7bfcd2c4e9d58408ec8c37e8293260dbe354b61140695111c529485cf6c809

Stefan Claas wrote:

> Correct! Now my base26 Encoder/Decoder handles Unicode.

$ echo -n '祝你新年快乐' | base26 | ug -g
XIJGB GUIHH EGWIU FUGVI DHYGV IJHPG UIDHO F

$ echo -n 'XIJGB GUIHH EGWIU FUGVI DHYGV IJHPG UIDHO F' | ug | base26 -d
祝你新年快乐

Regards
Stefan
--
----Ed25519 Signature----
2bf6d991aab695a1adaf9c2260a47a4dbc052fde2a12ab594620c0920dc8fafa
ba269c382eae31749cea21ecba0f3204529ad430b931e8ee1fc40de91710c403