Skip to content

Instantly share code, notes, and snippets.

@Ajnasz

Ajnasz/htmlentities.sed

Last active May 1, 2025 10:13
Show Gist options
  • Save Ajnasz/8d2ab713916be7790ebf303a5dd6f7d6 to your computer and use it in GitHub Desktop.
Save Ajnasz/8d2ab713916be7790ebf303a5dd6f7d6 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
htmlentities.sed
# SED Commands to replace HTML entities to characters
# Usage:
# sed -f htmlentities.sed file.html > newfile.html
# Replace entities in all html files
# sed -i -f htmlentities.sed file.html **/*.html
# ASCII printable characters
s/ / /g
s/!/!/g
s/"/"/g
s/"/"/g
s/#/#/g
s/$/$/g
s/%/%/g
s/&/\&/g
s/&/\&/g
s/'/'/g
s/(/(/g
s/)/)/g
s/*/*/g
s/+/+/g
s/,/,/g
s/-/-/g
s/././g
s///\//g
s/0/0/g
s/1/1/g
s/2/2/g
s/3/3/g
s/4/4/g
s/5/5/g
s/6/6/g
s/7/7/g
s/8/8/g
s/9/9/g
s/:/:/g
s/&#59;/;/g
s/&lt;/</g
s/&#60;/</g
s/&#61;/=/g
s/&gt;/>/g
s/&#62;/>/g
s/&#63;/?/g
s/&#64;/@/g
s/&#65;/A/g
s/&#66;/B/g
s/&#67;/C/g
s/&#68;/D/g
s/&#69;/E/g
s/&#70;/F/g
s/&#71;/G/g
s/&#72;/H/g
s/&#73;/I/g
s/&#74;/J/g
s/&#75;/K/g
s/&#76;/L/g
s/&#77;/M/g
s/&#78;/N/g
s/&#79;/O/g
s/&#80;/P/g
s/&#81;/Q/g
s/&#82;/R/g
s/&#83;/S/g
s/&#84;/T/g
s/&#85;/U/g
s/&#86;/V/g
s/&#87;/W/g
s/&#88;/X/g
s/&#89;/Y/g
s/&#90;/Z/g
s/&#91;/[/g
s/&#92;/\\/g
s/&#93;/]/g
s/&#94;/^/g
s/&#95;/_/g
s/&#96;/`/g
s/&#97;/a/g
s/&#98;/b/g
s/&#99;/c/g
s/&#100;/d/g
s/&#101;/e/g
s/&#102;/f/g
s/&#103;/g/g
s/&#104;/h/g
s/&#105;/i/g
s/&#106;/j/g
s/&#107;/k/g
s/&#108;/l/g
s/&#109;/m/g
s/&#110;/n/g
s/&#111;/o/g
s/&#112;/p/g
s/&#113;/q/g
s/&#114;/r/g
s/&#115;/s/g
s/&#116;/t/g
s/&#117;/u/g
s/&#118;/v/g
s/&#119;/w/g
s/&#120;/x/g
s/&#121;/y/g
s/&#122;/z/g
s/&#123;/{/g
s/&#124;/|/g
s/&#125;/}/g
s/&#126;/~/g
# ISO-8859-1 characters
s/&Agrave;/À/g
s/&#192;/À/g
s/&Aacute;/Á/g
s/&#193;/Á/g
s/&Acirc;/Â/g
s/&#194;/Â/g
s/&Atilde;/Ã/g
s/&#195;/Ã/g
s/&Auml;/Ä/g
s/&#196;/Ä/g
s/&Aring;/Å/g
s/&#197;/Å/g
s/&AElig;/Æ/g
s/&#198;/Æ/g
s/&Ccedil;/Ç/g
s/&#199;/Ç/g
s/&Egrave;/È/g
s/&#200;/È/g
s/&Eacute;/É/g
s/&#201;/É/g
s/&Ecirc;/Ê/g
s/&#202;/Ê/g
s/&Euml;/Ë/g
s/&#203;/Ë/g
s/&Igrave;/Ì/g
s/&#204;/Ì/g
s/&Iacute;/Í/g
s/&#205;/Í/g
s/&Icirc;/Î/g
s/&#206;/Î/g
s/&Iuml;/Ï/g
s/&#207;/Ï/g
s/&ETH;/Ð/g
s/&#208;/Ð/g
s/&Ntilde;/Ñ/g
s/&#209;/Ñ/g
s/&Ograve;/Ò/g
s/&#210;/Ò/g
s/&Oacute;/Ó/g
s/&#211;/Ó/g
s/&Ocirc;/Ô/g
s/&#212;/Ô/g
s/&Otilde;/Õ/g
s/&#213;/Õ/g
s/&Ouml;/Ö/g
s/&#214;/Ö/g
s/&Oslash;/Ø/g
s/&#216;/Ø/g
s/&Ugrave;/Ù/g
s/&#217;/Ù/g
s/&Uacute;/Ú/g
s/&#218;/Ú/g
s/&Ucirc;/Û/g
s/&#219;/Û/g
s/&Uuml;/Ü/g
s/&#220;/Ü/g
s/&Yacute;/Ý/g
s/&#221;/Ý/g
s/&THORN;/Þ/g
s/&#222;/Þ/g
s/&szlig;/ß/g
s/&#223;/ß/g
s/&agrave;/à/g
s/&#224;/à/g
s/&aacute;/á/g
s/&#225;/á/g
s/&acirc;/â/g
s/&#226;/â/g
s/&atilde;/ã/g
s/&#227;/ã/g
s/&auml;/ä/g
s/&#228;/ä/g
s/&aring;/å/g
s/&#229;/å/g
s/&aelig;/æ/g
s/&#230;/æ/g
s/&ccedil;/ç/g
s/&#231;/ç/g
s/&egrave;/è/g
s/&#232;/è/g
s/&eacute;/é/g
s/&#233;/é/g
s/&ecirc;/ê/g
s/&#234;/ê/g
s/&euml;/ë/g
s/&#235;/ë/g
s/&igrave;/ì/g
s/&#236;/ì/g
s/&iacute;/í/g
s/&#237;/í/g
s/&icirc;/î/g
s/&#238;/î/g
s/&iuml;/ï/g
s/&#239;/ï/g
s/&eth;/ð/g
s/&#240;/ð/g
s/&ntilde;/ñ/g
s/&#241;/ñ/g
s/&ograve;/ò/g
s/&#242;/ò/g
s/&oacute;/ó/g
s/&#243;/ó/g
s/&ocirc;/ô/g
s/&#244;/ô/g
s/&otilde;/õ/g
s/&#245;/õ/g
s/&ouml;/ö/g
s/&#246;/ö/g
s/&oslash;/ø/g
s/&#248;/ø/g
s/&ugrave;/ù/g
s/&#249;/ù/g
s/&uacute;/ú/g
s/&#250;/ú/g
s/&ucirc;/û/g
s/&#251;/û/g
s/&uuml;/ü/g
s/&#252;/ü/g
s/&yacute;/ý/g
s/&#253;/ý/g
s/&thorn;/þ/g
s/&#254;/þ/g
s/&yuml;/ÿ/g
s/&#255;/ÿ/g
# ISO-8859-1 symbols
s/&nbsp;/ /g
s/&#160;/ /g
s/&iexcl;/¡/g
s/&#161;/¡/g
s/&cent;/¢/g
s/&#162;/¢/g
s/&pound;/£/g
s/&#163;/£/g
s/&curren;/¤/g
s/&#164;/¤/g
s/&yen;/¥/g
s/&#165;/¥/g
s/&brvbar;/¦/g
s/&#166;/¦/g
s/&sect;/§/g
s/&#167;/§/g
s/&uml;/¨/g
s/&#168;/¨/g
s/&copy;/©/g
s/&#169;/©/g
s/&ordf;/ª/g
s/&#170;/ª/g
s/&laquo;/«/g
s/&#171;/«/g
s/&not;/¬/g
s/&#172;/¬/g
s/&shy;/­/g
s/&#173;/­/g
s/&reg;/®/g
s/&#174;/®/g
s/&macr;/¯/g
s/&#175;/¯/g
s/&deg;/°/g
s/&#176;/°/g
s/&plusmn;/±/g
s/&#177;/±/g
s/&sup2;/²/g
s/&#178;/²/g
s/&sup3;/³/g
s/&#179;/³/g
s/&acute;/´/g
s/&#180;/´/g
s/&micro;/µ/g
s/&#181;/µ/g
s/&para;//g
s/&#182;//g
s/&cedil;/¸/g
s/&#184;/¸/g
s/&sup1;/¹/g
s/&#185;/¹/g
s/&ordm;/º/g
s/&#186;/º/g
s/&raquo;/»/g
s/&#187;/»/g
s/&frac14;/¼/g
s/&#188;/¼/g
s/&frac12;/½/g
s/&#189;/½/g
s/&frac34;/¾/g
s/&#190;/¾/g
s/&iquest;/¿/g
s/&#191;/¿/g
s/&times;/×/g
s/&#215;/×/g
s/&divide;/÷/g
s/&#247;/÷/g
# Math symbols
s/&forall;/∀/g
s/&#8704;/∀/g
s/&part;/∂/g
s/&#8706;/∂/g
s/&exist;/∃/g
s/&#8707;/∃/g
s/&empty;/∅/g
s/&#8709;/∅/g
s/&nabla;/∇/g
s/&#8711;/∇/g
s/&isin;/∈/g
s/&#8712;/∈/g
s/&notin;/∉/g
s/&#8713;/∉/g
s/&ni;/∋/g
s/&#8715;/∋/g
s/&prod;/∏/g
s/&#8719;/∏/g
s/&sum;/∑/g
s/&#8721;/∑/g
s/&minus;/−/g
s/&#8722;/−/g
s/&lowast;/∗/g
s/&#8727;/∗/g
s/&radic;/√/g
s/&#8730;/√/g
s/&prop;/∝/g
s/&#8733;/∝/g
s/&infin;/∞/g
s/&#8734;/∞/g
s/&ang;/∠/g
s/&#8736;/∠/g
s/&and;/∧/g
s/&#8743;/∧/g
s/&or;/∨/g
s/&#8744;/∨/g
s/&cap;/∩/g
s/&#8745;/∩/g
s/&cup;/∪/g
s/&#8746;/∪/g
s/&int;/∫/g
s/&#8747;/∫/g
s/&there4;/∴/g
s/&#8756;/∴/g
s/&sim;/∼/g
s/&#8764;/∼/g
s/&cong;/≅/g
s/&#8773;/≅/g
s/&asymp;/≈/g
s/&#8776;/≈/g
s/&ne;/≠/g
s/&#8800;/≠/g
s/&equiv;/≡/g
s/&#8801;/≡/g
s/&le;/≤/g
s/&#8804;/≤/g
s/&ge;/≥/g
s/&#8805;/≥/g
s/&sub;/⊂/g
s/&#8834;/⊂/g
s/&sup;/⊃/g
s/&#8835;/⊃/g
s/&nsub;/⊄/g
s/&#8836;/⊄/g
s/&sube;/⊆/g
s/&#8838;/⊆/g
s/&supe;/⊇/g
s/&#8839;/⊇/g
s/&oplus;/⊕/g
s/&#8853;/⊕/g
s/&otimes;/⊗/g
s/&#8855;/⊗/g
s/&perp;/⊥/g
s/&#8869;/⊥/g
s/&sdot;/⋅/g
s/&#8901;/⋅/g
# Greek letters
s/&Alpha;/Α/g
s/&#913;/Α/g
s/&Beta;/Β/g
s/&#914;/Β/g
s/&Gamma;/Γ/g
s/&#915;/Γ/g
s/&Delta;/Δ/g
s/&#916;/Δ/g
s/&Epsilon;/Ε/g
s/&#917;/Ε/g
s/&Zeta;/Ζ/g
s/&#918;/Ζ/g
s/&Eta;/Η/g
s/&#919;/Η/g
s/&Theta;/Θ/g
s/&#920;/Θ/g
s/&Iota;/Ι/g
s/&#921;/Ι/g
s/&Kappa;/Κ/g
s/&#922;/Κ/g
s/&Lambda;/Λ/g
s/&#923;/Λ/g
s/&Mu;/Μ/g
s/&#924;/Μ/g
s/&Nu;/Ν/g
s/&#925;/Ν/g
s/&Xi;/Ξ/g
s/&#926;/Ξ/g
s/&Omicron;/Ο/g
s/&#927;/Ο/g
s/&Pi;/Π/g
s/&#928;/Π/g
s/&Rho;/Ρ/g
s/&#929;/Ρ/g
s/&Sigma;/Σ/g
s/&#931;/Σ/g
s/&Tau;/Τ/g
s/&#932;/Τ/g
s/&Upsilon;/Υ/g
s/&#933;/Υ/g
s/&Phi;/Φ/g
s/&#934;/Φ/g
s/&Chi;/Χ/g
s/&#935;/Χ/g
s/&Psi;/Ψ/g
s/&#936;/Ψ/g
s/&Omega;/Ω/g
s/&#937;/Ω/g
s/&alpha;/α/g
s/&#945;/α/g
s/&beta;/β/g
s/&#946;/β/g
s/&gamma;/γ/g
s/&#947;/γ/g
s/&delta;/δ/g
s/&#948;/δ/g
s/&epsilon;/ε/g
s/&#949;/ε/g
s/&zeta;/ζ/g
s/&#950;/ζ/g
s/&eta;/η/g
s/&#951;/η/g
s/&theta;/θ/g
s/&#952;/θ/g
s/&iota;/ι/g
s/&#953;/ι/g
s/&kappa;/κ/g
s/&#954;/κ/g
s/&lambda;/λ/g
s/&#955;/λ/g
s/&mu;/μ/g
s/&#956;/μ/g
s/&nu;/ν/g
s/&#957;/ν/g
s/&xi;/ξ/g
s/&#958;/ξ/g
s/&omicron;/ο/g
s/&#959;/ο/g
s/&pi;/π/g
s/&#960;/π/g
s/&rho;/ρ/g
s/&#961;/ρ/g
s/&sigmaf;/ς/g
s/&#962;/ς/g
s/&sigma;/σ/g
s/&#963;/σ/g
s/&tau;/τ/g
s/&#964;/τ/g
s/&upsilon;/υ/g
s/&#965;/υ/g
s/&phi;/φ/g
s/&#966;/φ/g
s/&chi;/χ/g
s/&#967;/χ/g
s/&psi;/ψ/g
s/&#968;/ψ/g
s/&omega;/ω/g
s/&#969;/ω/g
s/&thetasym;/ϑ/g
s/&#977;/ϑ/g
s/&upsih;/ϒ/g
s/&#978;/ϒ/g
s/&piv;/ϖ/g
s/&#982;/ϖ/g
# Misc entities
s/&OElig;/Œ/g
s/&#338;/Œ/g
s/&oelig;/œ/g
s/&#339;/œ/g
s/&Scaron;/Š/g
s/&#352;/Š/g
s/&scaron;/š/g
s/&#353;/š/g
s/&Yuml;/Ÿ/g
s/&#376;/Ÿ/g
s/&fnof;/ƒ/g
s/&#402;/ƒ/g
s/&circ;/ˆ/g
s/&#710;/ˆ/g
s/&tilde;/˜/g
s/&#732;/˜/g
s/&ensp;/ /g
s/&#8194;/ /g
s/&emsp;/ /g
s/&#8195;/ /g
s/&thinsp;/ /g
s/&#8201;/ /g
s/&zwnj;/‌/g
s/&#8204;/‌/g
s/&zwj;/‍/g
s/&#8205;/‍/g
s/&lrm;/‎/g
s/&#8206;/‎/g
s/&rlm;/‏/g
s/&#8207;/‏/g
s/&ndash;/–/g
s/&#8211;/–/g
s/&mdash;/—/g
s/&#8212;/—/g
s/&lsquo;/‘/g
s/&#8216;/‘/g
s/&rsquo;/’/g
s/&#8217;/’/g
s/&sbquo;/‚/g
s/&#8218;/‚/g
s/&ldquo;/“/g
s/&#8220;/“/g
s/&rdquo;/”/g
s/&#8221;/”/g
s/&bdquo;/„/g
s/&#8222;/„/g
s/&dagger;/†/g
s/&#8224;/†/g
s/&Dagger;/‡/g
s/&#8225;/‡/g
s/&bull;/•/g
s/&#8226;/•/g
s/&hellip;/…/g
s/&#8230;/…/g
s/&permil;/‰/g
s/&#8240;/‰/g
s/&prime;/′/g
s/&#8242;/′/g
s/&Prime;/″/g
s/&#8243;/″/g
s/&lsaquo;/‹/g
s/&#8249;/‹/g
s/&rsaquo;/›/g
s/&#8250;/›/g
s/&oline;/‾/g
s/&#8254;/‾/g
s/&euro;/€/g
s/&#8364;/€/g
s/&trade;/™/g
s/&#8482;/™/g
s/&larr;/←/g
s/&#8592;/←/g
s/&uarr;/↑/g
s/&#8593;/↑/g
s/&rarr;/→/g
s/&#8594;/→/g
s/&darr;/↓/g
s/&#8595;/↓/g
s/&harr;/↔/g
s/&#8596;/↔/g
s/&crarr;/↵/g
s/&#8629;/↵/g
s/&lceil;/⌈/g
s/&#8968;/⌈/g
s/&rceil;/⌉/g
s/&#8969;/⌉/g
s/&lfloor;/⌊/g
s/&#8970;/⌊/g
s/&rfloor;/⌋/g
s/&#8971;/⌋/g
s/&loz;/◊/g
s/&#9674;/◊/g
s/&spades;/♠/g
s/&#9824;/♠/g
s/&clubs;/♣/g
s/&#9827;/♣/g
s/&hearts;/♥/g
s/&#9829;/♥/g
s/&diams;/♦/g
s/&#9830;/♦/g
@robin-a-meade
Copy link

robin-a-meade commented Nov 22, 2022 •
edited
Loading

sed treats ampersand specially in the replacement. See https://unix.stackexchange.com/a/296732

Consider:

echo '&amp;' | sed -f htmlentities.sed

Result:

&amp;

Expected:

&

Need to change s/&amp;/&/g to s/&amp;/\&/g (must backslash escape the special meaning of ampersand)

Also: The replacement of ampersand must be the last replacement. That is, move s/&amp;/&/g to be the last line of the sed script.

Test using:

echo '&amp;lt;' | sed -f htmlentities.sed

The result should be:

&lt;

@Ajnasz
Copy link
Author

Ajnasz commented Nov 28, 2022

Thanks @robin-a-meade, I changed the gist according to your comment.

@robin-a-meade
Copy link

The script works well! Thanks @Ajnasz .

@Hangaro0
Copy link

Hangaro0 commented May 1, 2025 •
edited
Loading

s/&quot;/"/g missing to add

@Ajnasz
Copy link
Author

Ajnasz commented May 1, 2025

Hey @Hangaro0
Added it too, thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment