I've found that this issue was caused by the new LaTeX hook system, respective the fact that we redefined \@begindocumenthook to execute some code that fixes similar cases. I've modified tex4ht.sty to use new hooks and it seems to work correctly now.

reference:

https://tex.stackexchange.com/questions/576260/tex4ht-gives-catcode-error-when-loading-mathtools

I do not know if this is new error, caused by latest TL update. But this is first time I see it.

This MWE compiles with no problem with lualatex, but gives

(/usr/local/texlive/2020/texmf-dist/tex/latex/base/ts1cmr.fd) (/usr/local/texlive/2020/texmf-dist/tex/latex/graphics/graphicx.sty (/usr/local/texlive/2020/texmf-dist/tex/generic/tex4ht/graphics-hooks.4ht) (/usr/local/texlive/2020/texmf-dist/tex/latex/graphics/graphics.sty
! Missing number, treated as zero.
<to be read again>
^
l.39 \catcode`\^^
A=\catcode`\%
?

When compiled with tex4ht. This is the smallest example I could make it to show the error. The Latex is auto-generated.

------------
\documentclass[11pt]{article}
\usepackage{amsmath}
\usepackage{mathtools}

\begin{document}
\begin{align*}
\frac{1}{18} \, \sqrt{3} \log\left(\frac{{\left| 6 \, x - 2 \, \sqrt{3} \right|}}
{{\left| 6 \, x + 2 \, \sqrt{3} \right|}}\right) - \frac{1}{18} \, \sqrt{3}
\log\left(-\frac{{\left| -6 \, x - 8 \, \sqrt{3} + 6 \, \sqrt{x^{2} + 1} - \frac{6}{x - \sqrt{x^{2} + 1}} \right|}}
{2 \, {\left(3 \, x - 4 \, \sqrt{3} - 3 \, \sqrt{x^{2} + 1} + \frac{3}{x - \sqrt{x^{2} + 1}}\right)}}\right)
- \frac{4 \, {\left(x - \sqrt{x^{2} + 1} + \frac{1}{x - \sqrt{x^{2} + 1}}\right)}}
{3 \, {\left(3 \, {\left(x - \sqrt{x^{2} + 1} +
\frac{1}{x - \sqrt{x^{2} + 1}}\right)}^{2} - 16\right)}} - \frac{4 \, x}{3 \, {\left(3 \, x^{2} - 1\right)}}
\end{align*}
\end{document}
--------------------------

And now I compile using:

make4ht -ulm default -a debug foo5.tex "htm"
it gives

(base) >make4ht -ulm default -a debug foo5.tex "htm"
[INFO] mkparams: Output dir:
[INFO] mkparams: Compiler: dvilualatex
[INFO] mkparams: Latex options: -jobname=foo5
[INFO] mkparams: tex4ht.sty: xhtml,htm,charset=utf-8
[INFO] mkparams: tex4ht: -cmozhtf -utf8
[INFO] mkparams: build_file: foo5.mk4
[INFO] mkparams: Output format: html5
[STATUS] make4ht: Conversion started
[STATUS] make4ht: Input file: foo5.tex
[INFO] mkutils: Load extension common_domfilters
[INFO] mkutils: Cannot open config file foo5.mk4
[INFO] make4ht-lib: setting param correct_exit
[INFO] make4ht-lib: setting param correct_exit
[INFO] make4ht-lib: setting param correct_exit
[INFO] make4ht-lib: setting param ext
[INFO] make4ht-lib: Adding: ext dvi
[INFO] htlatex: LaTeX call: dvilualatex --interaction=errorstopmode -jobname=foo5 '\makeatletter\def\HCode{\futurelet\HCode\HChar}\def\HChar{\ifx"\HCode\def\HCode"##1"{\Link##1}\expandafter\HCode\else\expandafter\Link\fi}\def\Link#1.a.b.c.{\g@addto@macro\@documentclasshook{\RequirePackage[#1,html]{tex4ht}}\let\HCode\documentstyle\def\documentstyle{\let\documentstyle\HCode\expandafter\def\csname tex4ht\endcsname{#1,html}\def\HCode####1{\documentstyle[tex4ht,}\@ifnextchar[{\HCode}{\documentstyle[tex4ht]}}}\makeatother\HCode xhtml,htm,charset=utf-8,html5.a.b.c.\input "\detokenize{foo5.tex}"'
This is LuaTeX, Version 1.12.0 (TeX Live 2020)
restricted system commands enabled.
LaTeX2e <2020-10-01> patch level 2

L3 programming layer <2020-12-07> xparse <2020-03-03> (./foo5.tex (/usr/local/texlive/2020/texmf-dist/tex/latex/base/article.cls
Document Class: article 2020/04/10 v1.4m Standard LaTeX document class
(/usr/local/texlive/2020/texmf-dist/tex/latex/base/size11.clo)) (/usr/local/texlive/2020/texmf-dist/tex/generic/tex4ht/tex4ht.sty) (/usr/local/texlive/2020/texmf-dist/tex/generic/tex4ht/usepackage.4ht) (/usr/local/texlive/2020/texmf-dist/tex/latex/amsmath/amsmath.sty
For additional information on amsmath, use the `?' option.
(/usr/local/texlive/2020/texmf-dist/tex/latex/amsmath/amstext.sty (/usr/local/texlive/2020/texmf-dist/tex/latex/amsmath/amsgen.sty)) (/usr/local/texlive/2020/texmf-dist/tex/latex/amsmath/amsbsy.sty) (/usr/local/texlive/2020/texmf-dist/tex/latex/amsmath/amsopn.sty)) (/usr/local/texlive/2020/texmf-dist/tex/latex/mathtools/mathtools.sty (/usr/local/texlive/2020/texmf-dist/tex/latex/graphics/keyval.sty) (/usr/local/texlive/2020/texmf-dist/tex/latex/tools/calc.sty) (/usr/local/texlive/2020/texmf-dist/tex/latex/mathtools/mhsetup.sty)) (/usr/local/texlive/2020/texmf-dist/tex/generic/tex4ht/tex4ht.4ht
::::::::::::::::::::::::::::::::::::::::::
TeX4ht info is available in the log file
::::::::::::::::::::::::::::::::::::::::::
) (/usr/local/texlive/2020/texmf-dist/tex/generic/tex4ht/tex4ht.sty
--- needs --- tex4ht foo5 ---
(./foo5.tmp) (./foo5.xref) (/usr/local/texlive/2020/texmf-dist/tex/generic/tex4ht/html4.4ht) (/usr/local/texlive/2020/texmf-dist/tex/generic/tex4ht/unicode.4ht) (/usr/local/texlive/2020/texmf-dist/tex/generic/tex4ht/html4-math.4ht) (/usr/local/texlive/2020/texmf-dist/tex/generic/tex4ht/html5.4ht) (/usr/local/texlive/2020/texmf-dist/tex/generic/tex4ht/html4.4ht) (/usr/local/texlive/2020/texmf-dist/tex/generic/tex4ht/unicode.4ht) (/usr/local/texlive/2020/texmf-dist/tex/generic/tex4ht/html4-math.4ht) (/usr/local/texlive/2020/texmf-dist/tex/generic/tex4ht/html5.4ht) (/usr/local/texlive/2020/texmf-dist/tex/generic/tex4ht/latex.4ht (/usr/local/texlive/2020/texmf-dist/tex/latex/base/fontenc.sty (/usr/local/texlive/2020/texmf-dist/tex/latex/lm/ot1lmr.fd)) (/usr/local/texlive/2020/texmf-dist/tex/generic/kastrup/binhex.tex) (/usr/local/texlive/2020/texmf-dist/tex/latex/base/tuenc.def) (/usr/local/texlive/2020/texmf-dist/tex/generic/tex4ht/tuenc-luatex.4ht
(/usr/local/texlive/2020/texmf-dist/tex/luatex/luatexbase/luatexbase.sty (/usr/local/texlive/2020/texmf-dist/tex/luatex/ctablestack/ctablestack.sty)) (/usr/local/texlive/2020/texmf-dist/tex/lualatex/luacode/luacode.sty (/usr/local/texlive/2020/texmf-dist/tex/generic/iftex/ifluatex.sty (/usr/local/texlive/2020/texmf-dist/tex/generic/iftex/iftex.sty))) (/usr/local/texlive/2020/texmf-dist/tex/generic/tex4ht/html4.4ht) (/usr/local/texlive/2020/texmf-dist/tex/generic/tex4ht/unicode.4ht) (/usr/local/texlive/2020/texmf-dist/tex/generic/tex4ht/html4-math.4ht) (/usr/local/texlive/2020/texmf-dist/tex/generic/tex4ht/html5.4ht)) (/usr/local/texlive/2020/texmf-dist/tex/generic/tex4ht/html4.4ht) (/usr/local/texlive/2020/texmf-dist/tex/generic/tex4ht/unicode.4ht) (/usr/local/texlive/2020/texmf-dist/tex/generic/tex4ht/html4-math.4ht) (/usr/local/texlive/2020/texmf-dist/tex/generic/tex4ht/html5.4ht) (/usr/local/texlive/2020/texmf-dist/tex/generic/tex4ht/html4.4ht)
(/usr/local/texlive/2020/texmf-dist/tex/generic/tex4ht/unicode.4ht) (/usr/local/texlive/2020/texmf-dist/tex/generic/tex4ht/html4-math.4ht) (/usr/local/texlive/2020/texmf-dist/tex/generic/tex4ht/html5.4ht)) (/usr/local/texlive/2020/texmf-dist/tex/generic/tex4ht/fontmath.4ht (/usr/local/texlive/2020/texmf-dist/tex/generic/tex4ht/html4.4ht) (/usr/local/texlive/2020/texmf-dist/tex/generic/tex4ht/unicode.4ht) (/usr/local/texlive/2020/texmf-dist/tex/generic/tex4ht/html4-math.4ht) (/usr/local/texlive/2020/texmf-dist/tex/generic/tex4ht/html5.4ht)) (/usr/local/texlive/2020/texmf-dist/tex/generic/tex4ht/article.4ht (/usr/local/texlive/2020/texmf-dist/tex/generic/tex4ht/html4.4ht) (/usr/local/texlive/2020/texmf-dist/tex/generic/tex4ht/unicode.4ht) (/usr/local/texlive/2020/texmf-dist/tex/generic/tex4ht/html4-math.4ht) (/usr/local/texlive/2020/texmf-dist/tex/generic/tex4ht/html5.4ht)) (/usr/local/texlive/2020/texmf-dist/tex/generic/tex4ht/amsmath.4ht
(/usr/local/texlive/2020/texmf-dist/tex/generic/tex4ht/html4.4ht) (/usr/local/texlive/2020/texmf-dist/tex/generic/tex4ht/unicode.4ht) (/usr/local/texlive/2020/texmf-dist/tex/generic/tex4ht/html4-math.4ht) (/usr/local/texlive/2020/texmf-dist/tex/generic/tex4ht/html5.4ht)) (/usr/local/texlive/2020/texmf-dist/tex/generic/tex4ht/amstext.4ht (/usr/local/texlive/2020/texmf-dist/tex/generic/tex4ht/html4.4ht) (/usr/local/texlive/2020/texmf-dist/tex/generic/tex4ht/unicode.4ht) (/usr/local/texlive/2020/texmf-dist/tex/generic/tex4ht/html4-math.4ht) (/usr/local/texlive/2020/texmf-dist/tex/generic/tex4ht/html5.4ht)) (/usr/local/texlive/2020/texmf-dist/tex/generic/tex4ht/amsbsy.4ht (/usr/local/texlive/2020/texmf-dist/tex/generic/tex4ht/html4.4ht) (/usr/local/texlive/2020/texmf-dist/tex/generic/tex4ht/unicode.4ht) (/usr/local/texlive/2020/texmf-dist/tex/generic/tex4ht/html4-math.4ht) (/usr/local/texlive/2020/texmf-dist/tex/generic/tex4ht/html5.4ht))
(/usr/local/texlive/2020/texmf-dist/tex/generic/tex4ht/amsopn.4ht (/usr/local/texlive/2020/texmf-dist/tex/generic/tex4ht/html4.4ht) (/usr/local/texlive/2020/texmf-dist/tex/generic/tex4ht/unicode.4ht) (/usr/local/texlive/2020/texmf-dist/tex/generic/tex4ht/html4-math.4ht) (/usr/local/texlive/2020/texmf-dist/tex/generic/tex4ht/html5.4ht)) (/usr/local/texlive/2020/texmf-dist/tex/generic/tex4ht/mathtools.4ht (/usr/local/texlive/2020/texmf-dist/tex/generic/tex4ht/html4.4ht) (/usr/local/texlive/2020/texmf-dist/tex/generic/tex4ht/unicode.4ht) (/usr/local/texlive/2020/texmf-dist/tex/generic/tex4ht/html4-math.4ht) (/usr/local/texlive/2020/texmf-dist/tex/generic/tex4ht/html5.4ht)) (/usr/local/texlive/2020/texmf-dist/tex/generic/tex4ht/html4.4ht) (/usr/local/texlive/2020/texmf-dist/tex/generic/tex4ht/unicode.4ht) (/usr/local/texlive/2020/texmf-dist/tex/generic/tex4ht/html4-math.4ht) (/usr/local/texlive/2020/texmf-dist/tex/generic/tex4ht/html5.4ht))
(/usr/local/texlive/2020/texmf-dist/tex/latex/l3backend/l3backend-dvips.def) (./foo5.aux) (/usr/local/texlive/2020/texmf-dist/tex/latex/base/ts1cmr.fd) (/usr/local/texlive/2020/texmf-dist/tex/latex/graphics/graphicx.sty (/usr/local/texlive/2020/texmf-dist/tex/generic/tex4ht/graphics-hooks.4ht) (/usr/local/texlive/2020/texmf-dist/tex/latex/graphics/graphics.sty
! Missing number, treated as zero.
<to be read again>
^
l.39 \catcode`\^^
A=\catcode`\%
?

------------------------

When removing the \usepackage{mathtools} package, it compiles OK. No errors. TL fully updated 3-4 days ago.

make4ht --version
make4ht version v0.3f

which tex4ht
/usr/local/texlive/2020/bin/x86_64-linux/tex4ht