Rocksolid Light

Welcome to RetroBBS

mail  files  register  newsreader  groups  login

Message-ID:  

<Stealth> How do I bind a computer to an NIS server? <Joey> Use a rope? -- Seen on #Debian


devel / comp.compression / Additions to MTF?

SubjectAuthor
* Additions to MTF?Harry Potter
`- Re: Additions to MTF?Harry Potter

1
Additions to MTF?

<c417a1dc-35c5-445e-9798-2411dd47cefdn@googlegroups.com>

  copy mid

https://www.rocksolidbbs.com/devel/article-flat.php?id=1207&group=comp.compression#1207

  copy link   Newsgroups: comp.compression
X-Received: by 2002:a05:620a:140b:b0:76d:ada1:d841 with SMTP id d11-20020a05620a140b00b0076dada1d841mr108876qkj.7.1697906022805;
Sat, 21 Oct 2023 09:33:42 -0700 (PDT)
X-Received: by 2002:a05:6808:1a06:b0:3a8:74ff:6c01 with SMTP id
bk6-20020a0568081a0600b003a874ff6c01mr1933150oib.5.1697906022650; Sat, 21 Oct
2023 09:33:42 -0700 (PDT)
Path: i2pn2.org!i2pn.org!news.1d4.us!usenet.blueworldhosting.com!diablo1.usenet.blueworldhosting.com!1.us.feeder.erje.net!3.us.feeder.erje.net!feeder.erje.net!border-1.nntp.ord.giganews.com!nntp.giganews.com!news-out.google.com!nntp.google.com!postnews.google.com!google-groups.googlegroups.com!not-for-mail
Newsgroups: comp.compression
Date: Sat, 21 Oct 2023 09:33:42 -0700 (PDT)
Injection-Info: google-groups.googlegroups.com; posting-host=173.68.2.174; posting-account=xRocggoAAACFej4w6sQauoZjUP9yroE5
NNTP-Posting-Host: 173.68.2.174
User-Agent: G2/1.0
MIME-Version: 1.0
Message-ID: <c417a1dc-35c5-445e-9798-2411dd47cefdn@googlegroups.com>
Subject: Additions to MTF?
From: rose.joseph12@yahoo.com (Harry Potter)
Injection-Date: Sat, 21 Oct 2023 16:33:42 +0000
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Lines: 8
 by: Harry Potter - Sat, 21 Oct 2023 16:33 UTC

Hi! Recently, I found some bugs in my rendition of Adaptive Huffman codes, and the bug fixes cost me big time. :( After reimplementing and tweaking MTF, I found it to be better than Adaptive Huffman, but I'm not totally sure yet, as I need to debug it. Right now, the 16-bit version seems to be doing about 4.6% better than 7Zip, but I have two problem test files--and one of them is uncompressible using my technique but gives me about 7.8% using Deflate--and am still .4% off my long-term goal there. I had to kill some methods in the process. :( Is there anything I can add to MTF to make it better? Thank you.

Re: Additions to MTF?

<eab6a1f7-3d81-4c2b-a00c-d9a2ba83b6e7n@googlegroups.com>

  copy mid

https://www.rocksolidbbs.com/devel/article-flat.php?id=1208&group=comp.compression#1208

  copy link   Newsgroups: comp.compression
X-Received: by 2002:ac8:480a:0:b0:41b:82ca:3aae with SMTP id g10-20020ac8480a000000b0041b82ca3aaemr110070qtq.2.1697925486163;
Sat, 21 Oct 2023 14:58:06 -0700 (PDT)
X-Received: by 2002:a05:6870:8a1e:b0:1e9:c362:a397 with SMTP id
p30-20020a0568708a1e00b001e9c362a397mr2471341oaq.10.1697925485902; Sat, 21
Oct 2023 14:58:05 -0700 (PDT)
Path: i2pn2.org!i2pn.org!usenet.blueworldhosting.com!diablo1.usenet.blueworldhosting.com!peer02.iad!feed-me.highwinds-media.com!news.highwinds-media.com!news-out.google.com!nntp.google.com!postnews.google.com!google-groups.googlegroups.com!not-for-mail
Newsgroups: comp.compression
Date: Sat, 21 Oct 2023 14:58:05 -0700 (PDT)
In-Reply-To: <c417a1dc-35c5-445e-9798-2411dd47cefdn@googlegroups.com>
Injection-Info: google-groups.googlegroups.com; posting-host=173.68.2.174; posting-account=xRocggoAAACFej4w6sQauoZjUP9yroE5
NNTP-Posting-Host: 173.68.2.174
References: <c417a1dc-35c5-445e-9798-2411dd47cefdn@googlegroups.com>
User-Agent: G2/1.0
MIME-Version: 1.0
Message-ID: <eab6a1f7-3d81-4c2b-a00c-d9a2ba83b6e7n@googlegroups.com>
Subject: Re: Additions to MTF?
From: rose.joseph12@yahoo.com (Harry Potter)
Injection-Date: Sat, 21 Oct 2023 21:58:06 +0000
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
X-Received-Bytes: 2178
 by: Harry Potter - Sat, 21 Oct 2023 21:58 UTC

Good news: I'm doing 5.4% better than 7Zip now. :) Bad news: the problem files are still there. :( This particular technique is to be close-source for a while after publication, but I'm willing to reveal the following in the hopes that somebody here can help me fix the problem files:

* I use Last16, where a recent LZ77 block is shortened to how many blocks ago was a match.
* I use my own version of BPE.
* I am using an MTF variant where the output is separated into ranges such that small numbers take fewer bits
* I modified Deflate's lazy mechanism such that, if compressing a two-byte LZ77 reference, the next block has to do at least two bytes better before skipping the current byte.
* I have a way to optimize values that are not in the range of a power of 2.

When I look at the problem test file that is uncompressible with VBinDiff, I find at the beginning of the file a lot of 0's, so it should be more compressible. I don't know why it's not compressing.

1
server_pubkey.txt

rocksolid light 0.9.8
clearnet tor