different-order entropy coding

Dear all, I am doing a project about compressing text files using different-order entropies. I am done with the calculating 1-order entropy P(xn), 2-order entropy (P(xn|xn-1)) and 3-order entropy (P(xn|xn-1,xn-2)). Here are my entropies 4.2019, 3.2661 and 2.379, respectively for a sample input text from Smaple text . In the second step, I wrote another code to encode 1-order entropy using Huffman Coding. It works and Decoding is giving me the same input. However, I am little bit confused for the 2-order and 3-order entropy. After calculating entropy, what is the next step? Here are my symbols for the 2-order entropy.

 ,
 -
 .
 :
 ;
 A
 B
 E
 F
 G
 H
 I
 L
 M
 O
 S
 T
 a
 b
 c
 d
 e
 f
 g
 h
 i
 j
 k
 l
 m
 n
 o
 p
 q
 r
 s
 t
 u
 v
 w
 x
 y
, 
,,
,-
,.
,:
,;
,A
,B
,E
,F
,G
,H
,I
,L
,M
,O
,S
,T
,a
,b
,c
,d
,e
,f
,g
,h
,i
,j
,k
,l
,m
,n
,o
,p
,q
,r
,s
,t
,u
,v
,w
,x
,y
- 
-,
--
-.
-:
-;
-A
-B
-E
-F
-G
-H
-I
-L
-M
-O
-S
-T
-a
-b
-c
-d
-e
-f
-g
-h
-i
-j
-k
-l
-m
-n
-o
-p
-q
-r
-s
-t
-u
-v
-w
-x
-y
. 
.,
.-
..
.:
.;
.A
.B
.E
.F
.G
.H
.I
.L
.M
.O
.S
.T
.a
.b
.c
.d
.e
.f
.g
.h
.i
.j
.k
.l
.m
.n
.o
.p
.q
.r
.s
.t
.u
.v
.w
.x
.y
: 
:,
:-
:.
::
:;
:A
:B
:E
:F
:G
:H
:I
:L
:M
:O
:S
:T
:a
:b
:c
:d
:e
:f
:g
:h
:i
:j
:k
:l
:m
:n
:o
:p
:q
:r
:s
:t
:u
:v
:w
:x
:y
; 
;,
;-
;.
;:
;;
;A
;B
;E
;F
;G
;H
;I
;L
;M
;O
;S
;T
;a
;b
;c
;d
;e
;f
;g
;h
;i
;j
;k
;l
;m
;n
;o
;p
;q
;r
;s
;t
;u
;v
;w
;x
;y
A 
A,
A-
A.
A:
A;
AA
AB
AE
AF
AG
AH
AI
AL
AM
AO
AS
AT
Aa
Ab
Ac
Ad
Ae
Af
Ag
Ah
Ai
Aj
Ak
Al
Am
An
Ao
Ap
Aq
Ar
As
At
Au
Av
Aw
Ax
Ay
B 
B,
B-
B.
B:
B;
BA
BB
BE
BF
BG
BH
BI
BL
BM
BO
BS
BT
Ba
Bb
Bc
Bd
Be
Bf
Bg
Bh
Bi
Bj
Bk
Bl
Bm
Bn
Bo
Bp
Bq
Br
Bs
Bt
Bu
Bv
Bw
Bx
By
E 
E,
E-
E.
E:
E;
EA
EB
EE
EF
EG
EH
EI
EL
EM
EO
ES
ET
Ea
Eb
Ec
Ed
Ee
Ef
Eg
Eh
Ei
Ej
Ek
El
Em
En
Eo
Ep
Eq
Er
Es
Et
Eu
Ev
Ew
Ex
Ey
F 
F,
F-
F.
F:
F;
FA
FB
FE
FF
FG
FH
FI
FL
FM
FO
FS
FT
Fa
Fb
Fc
Fd
Fe
Ff
Fg
Fh
Fi
Fj
Fk
Fl
Fm
Fn
Fo
Fp
Fq
Fr
Fs
Ft
Fu
Fv
Fw
Fx
Fy
G 
G,
G-
G.
G:
G;
GA
GB
GE
GF
GG
GH
GI
GL
GM
GO
GS
GT
Ga
Gb
Gc
Gd
Ge
Gf
Gg
Gh
Gi
Gj
Gk
Gl
Gm
Gn
Go
Gp
Gq
Gr
Gs
Gt
Gu
Gv
Gw
Gx
Gy
H 
H,
H-
H.
H:
H;
HA
HB
HE
HF
HG
HH
HI
HL
HM
HO
HS
HT
Ha
Hb
Hc
Hd
He
Hf
Hg
Hh
Hi
Hj
Hk
Hl
Hm
Hn
Ho
Hp
Hq
Hr
Hs
Ht
Hu
Hv
Hw
Hx
Hy
I 
I,
I-
I.
I:
I;
IA
IB
IE
IF
IG
IH
II
IL
IM
IO
IS
IT
Ia
Ib
Ic
Id
Ie
If
Ig
Ih
Ii
Ij
Ik
Il
Im
In
Io
Ip
Iq
Ir
Is
It
Iu
Iv
Iw
Ix
Iy
L 
L,
L-
L.
L:
L;
LA
LB
LE
LF
LG
LH
LI
LL
LM
LO
LS
LT
La
Lb
Lc
Ld
Le
Lf
Lg
Lh
Li
Lj
Lk
Ll
Lm
Ln
Lo
Lp
Lq
Lr
Ls
Lt
Lu
Lv
Lw
Lx
Ly
M 
M,
M-
M.
M:
M;
MA
MB
ME
MF
MG
MH
MI
ML
MM
MO
MS
MT
Ma
Mb
Mc
Md
Me
Mf
Mg
Mh
Mi
Mj
Mk
Ml
Mm
Mn
Mo
Mp
Mq
Mr
Ms
Mt
Mu
Mv
Mw
Mx
My
O 
O,
O-
O.
O:
O;
OA
OB
OE
OF
OG
OH
OI
OL
OM
OO
OS
OT
Oa
Ob
Oc
Od
Oe
Of
Og
Oh
Oi
Oj
Ok
Ol
Om
On
Oo
Op
Oq
Or
Os
Ot
Ou
Ov
Ow
Ox
Oy
S 
S,
S-
S.
S:
S;
SA
SB
SE
SF
SG
SH
SI
SL
SM
SO
SS
ST
Sa
Sb
Sc
Sd
Se
Sf
Sg
Sh
Si
Sj
Sk
Sl
Sm
Sn
So
Sp
Sq
Sr
Ss
St
Su
Sv
Sw
Sx
Sy
T 
T,
T-
T.
T:
T;
TA
TB
TE
TF
TG
TH
TI
TL
TM
TO
TS
TT
Ta
Tb
Tc
Td
Te
Tf
Tg
Th
Ti
Tj
Tk
Tl
Tm
Tn
To
Tp
Tq
Tr
Ts
Tt
Tu
Tv
Tw
Tx
Ty
a 
a,
a-
a.
a:
a;
aA
aB
aE
aF
aG
aH
aI
aL
aM
aO
aS
aT
aa
ab
ac
ad
ae
af
ag
ah
ai
aj
ak
al
am
an
ao
ap
aq
ar
as
at
au
av
aw
ax
ay
b 
b,
b-
b.
b:
b;
bA
bB
bE
bF
bG
bH
bI
bL
bM
bO
bS
bT
ba
bb
bc
bd
be
bf
bg
bh
bi
bj
bk
bl
bm
bn
bo
bp
bq
br
bs
bt
bu
bv
bw
bx
by
c 
c,
c-
c.
c:
c;
cA
cB
cE
cF
cG
cH
cI
cL
cM
cO
cS
cT
ca
cb
cc
cd
ce
cf
cg
ch
ci
cj
ck
cl
cm
cn
co
cp
cq
cr
cs
ct
cu
cv
cw
cx
cy
d 
d,
d-
d.
d:
d;
dA
dB
dE
dF
dG
dH
dI
dL
dM
dO
dS
dT
da
db
dc
dd
de
df
dg
dh
di
dj
dk
dl
dm
dn
do
dp
dq
dr
ds
dt
du
dv
dw
dx
dy
e 
e,
e-
e.
e:
e;
eA
eB
eE
eF
eG
eH
eI
eL
eM
eO
eS
eT
ea
eb
ec
ed
ee
ef
eg
eh
ei
ej
ek
el
em
en
eo
ep
eq
er
es
et
eu
ev
ew
ex
ey
f 
f,
f-
f.
f:
f;
fA
fB
fE
fF
fG
fH
fI
fL
fM
fO
fS
fT
fa
fb
fc
fd
fe
ff
fg
fh
fi
fj
fk
fl
fm
fn
fo
fp
fq
fr
fs
ft
fu
fv
fw
fx
fy
g 
g,
g-
g.
g:
g;
gA
gB
gE
gF
gG
gH
gI
gL
gM
gO
gS
gT
ga
gb
gc
gd
ge
gf
gg
gh
gi
gj
gk
gl
gm
gn
go
gp
gq
gr
gs
gt
gu
gv
gw
gx
gy
h 
h,
h-
h.
h:
h;
hA
hB
hE
hF
hG
hH
hI
hL
hM
hO
hS
hT
ha
hb
hc
hd
he
hf
hg
hh
hi
hj
hk
hl
hm
hn
ho
hp
hq
hr
hs
ht
hu
hv
hw
hx
hy
i 
i,
i-
i.
i:
i;
iA
iB
iE
iF
iG
iH
iI
iL
iM
iO
iS
iT
ia
ib
ic
id
ie
if
ig
ih
ii
ij
ik
il
im
in
io
ip
iq
ir
is
it
iu
iv
iw
ix
iy
j 
j,
j-
j.
j:
j;
jA
jB
jE
jF
jG
jH
jI
jL
jM
jO
jS
jT
ja
jb
jc
jd
je
jf
jg
jh
ji
jj
jk
jl
jm
jn
jo
jp
jq
jr
js
jt
ju
jv
jw
jx
jy
k 
k,
k-
k.
k:
k;
kA
kB
kE
kF
kG
kH
kI
kL
kM
kO
kS
kT
ka
kb
kc
kd
ke
kf
kg
kh
ki
kj
kk
kl
km
kn
ko
kp
kq
kr
ks
kt
ku
kv
kw
kx
ky
l 
l,
l-
l.
l:
l;
lA
lB
lE
lF
lG
lH
lI
lL
lM
lO
lS
lT
la
lb
lc
ld
le
lf
lg
lh
li
lj
lk
ll
lm
ln
lo
lp
lq
lr
ls
lt
lu
lv
lw
lx
ly
m 
m,
m-
m.
m:
m;
mA
mB
mE
mF
mG
mH
mI
mL
mM
mO
mS
mT
ma
mb
mc
md
me
mf
mg
mh
mi
mj
mk
ml
mm
mn
mo
mp
mq
mr
ms
mt
mu
mv
mw
mx
my
n 
n,
n-
n.
n:
n;
nA
nB
nE
nF
nG
nH
nI
nL
nM
nO
nS
nT
na
nb
nc
nd
ne
nf
ng
nh
ni
nj
nk
nl
nm
nn
no
np
nq
nr
ns
nt
nu
nv
nw
nx
ny
o 
o,
o-
o.
o:
o;
oA
oB
oE
oF
oG
oH
oI
oL
oM
oO
oS
oT
oa
ob
oc
od
oe
of
og
oh
oi
oj
ok
ol
om
on
oo
op
oq
or
os
ot
ou
ov
ow
ox
oy
p 
p,
p-
p.
p:
p;
pA
pB
pE
pF
pG
pH
pI
pL
pM
pO
pS
pT
pa
pb
pc
pd
pe
pf
pg
ph
pi
pj
pk
pl
pm
pn
po
pp
pq
pr
ps
pt
pu
pv
pw
px
py
q 
q,
q-
q.
q:
q;
qA
qB
qE
qF
qG
qH
qI
qL
qM
qO
qS
qT
qa
qb
qc
qd
qe
qf
qg
qh
qi
qj
qk
ql
qm
qn
qo
qp
qq
qr
qs
qt
qu
qv
qw
qx
qy
r 
r,
r-
r.
r:
r;
rA
rB
rE
rF
rG
rH
rI
rL
rM
rO
rS
rT
ra
rb
rc
rd
re
rf
rg
rh
ri
rj
rk
rl
rm
rn
ro
rp
rq
rr
rs
rt
ru
rv
rw
rx
ry
s 
s,
s-
s.
s:
s;
sA
sB
sE
sF
sG
sH
sI
sL
sM
sO
sS
sT
sa
sb
sc
sd
se
sf
sg
sh
si
sj
sk
sl
sm
sn
so
sp
sq
sr
ss
st
su
sv
sw
sx
sy
t 
t,
t-
t.
t:
t;
tA
tB
tE
tF
tG
tH
tI
tL
tM
tO
tS
tT
ta
tb
tc
td
te
tf
tg
th
ti
tj
tk
tl
tm
tn
to
tp
tq
tr
ts
tt
tu
tv
tw
tx
ty
u 
u,
u-
u.
u:
u;
uA
uB
uE
uF
uG
uH
uI
uL
uM
uO
uS
uT
ua
ub
uc
ud
ue
uf
ug
uh
ui
uj
uk
ul
um
un
uo
up
uq
ur
us
ut
uu
uv
uw
ux
uy
v 
v,
v-
v.
v:
v;
vA
vB
vE
vF
vG
vH
vI
vL
vM
vO
vS
vT
va
vb
vc
vd
ve
vf
vg
vh
vi
vj
vk
vl
vm
vn
vo
vp
vq
vr
vs
vt
vu
vv
vw
vx
vy
w 
w,
w-
w.
w:
w;
wA
wB
wE
wF
wG
wH
wI
wL
wM
wO
wS
wT
wa
wb
wc
wd
we
wf
wg
wh
wi
wj
wk
wl
wm
wn
wo
wp
wq
wr
ws
wt
wu
wv
ww
wx
wy
x 
x,
x-
x.
x:
x;
xA
xB
xE
xF
xG
xH
xI
xL
xM
xO
xS
xT
xa
xb
xc
xd
xe
xf
xg
xh
xi
xj
xk
xl
xm
xn
xo
xp
xq
xr
xs
xt
xu
xv
xw
xx
xy
y 
y,
y-
y.
y:
y;
yA
yB
yE
yF
yG
yH
yI
yL
yM
yO
yS
yT
ya
yb
yc
yd
ye
yf
yg
yh
yi
yj
yk
yl
ym
yn
yo
yp
yq
yr
ys
yt
yu
yv
yw
yx
yy
.
Thank you