Replacing special character 'É' to 'E'

52 views (last 30 days)
Pete sherer
Pete sherer on 28 Nov 2022
Edited: Stephen23 on 14 Dec 2023
Hi,
Is there a Matlab function to replace the special characters (like 'É') to the regular UTF-8 or ISO-8859-1?
Thanks,
  1 Comment
Stephen23
Stephen23 on 28 Nov 2022
"regular UTF-8 or ISO-8859-1"
Both UTF-8 (encodes all Unicode characters) and ISO-8859-1 include "É"... Perhaps you meant to ask something like "how to remove diacritics from characters?", which would match your question title.

Sign in to comment.

Accepted Answer

Jonas
Jonas on 28 Nov 2022
looks like there are only manual solutions.
Stackoverflow is your friend ;-)
  6 Comments
Jonas
Jonas on 29 Nov 2022
also it is qeustionable to do this whole thing since the change of letters can change th emeaning of the words, also in German for example, ä, ö and ü are changed to ae, oe and ue, but the same procedure does not make sence in other languages like turkish
Stephen23
Stephen23 on 12 Dec 2023
Edited: Stephen23 on 14 Dec 2023
@Jonas: your concern is well-founded. That function confuses two related (yet distinct) aspects of languages:
Note that ligatures are not diacritics, so splitting the ligatures Æ,Œ, etc. is not removing diacritics. The esszett character ß also does not have any diacritics nor is it considered to be a ligature (although it does derive from one). The lexicographical sorting rules of some languages do require treating those ligatures and characters as being equivalent to some other characters.... but that is distinct from removing diacritics from characters.
The function also fails to remove diacritics from other (even Latin-based) characters, e.g. Ǣ.
The function also returns the wrong character in some cases, e.g. eth ð has no diacritic. That it is commonly transliterated into latin script as d is irrelevant (and misleading: the digraph th would be better).
In short: the function is misnamed and does not really do what it claims.

Sign in to comment.

More Answers (2)

Stephen23
Stephen23 on 28 Nov 2022
Edited: Stephen23 on 28 Nov 2022
"Is there a Matlab function to replace the special characters (like 'É')"
You can call Python from MATLAB, and it can do the heavy-lifting:
inp = 'É';
baz = @(v)char(v(1)); % only need the first decomposed character.
out = baz(py.unicodedata.normalize('NFKD',inp)) % to remove diacritics.
out = 'E'
Read more:

John D'Errico
John D'Errico on 28 Nov 2022
Edited: John D'Errico on 28 Nov 2022
Easy peasy.
str = 'ABCDEFGHIJKÉÉÀÀÄÄabcdefghijkl'
str = 'ABCDEFGHIJKÉÉÀÀÄÄabcdefghijkl'
strrep(str,'É','E')
ans = 'ABCDEFGHIJKEEÀÀÄÄabcdefghijkl'
If there are other special characters you want replaced, strrep will handle them too, but it looks like you would need to do them one at a time with strrep. But other tools would certainly work too. Certainly regexp, but I've never been very good at regular expressions. :) This will work though:
badchar = 'ÉÀÄ';
goodchar = 'EAA';
[u,v] = ismember(str,'ÉÀÄ');
str(u) = goodchar(v(u))
str = 'ABCDEFGHIJKEEAAAAabcdefghijkl'
  1 Comment
Robert Wagner
Robert Wagner on 12 Dec 2023
but I've never been very good at regular expressions. :) ---> I've never tried to be in the first place... :-)))

Sign in to comment.

Categories

Find more on Characters and Strings in Help Center and File Exchange

Products


Release

R2022a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!