Created
January 22, 2016 07:09
-
-
Save ladder1984/0fe62194b04560768ccb to your computer and use it in GitHub Desktop.
过滤非BMP字符
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| def filter_invalid_str(text): | |
| """ | |
| 过滤非BMP字符 | |
| """ | |
| try: | |
| # UCS-4 | |
| highpoints = re.compile(u'[\U00010000-\U0010ffff]') | |
| except re.error: | |
| # UCS-2 | |
| highpoints = re.compile(u'[\uD800-\uDBFF][\uDC00-\uDFFF]') | |
| return highpoints.sub(u'_', text) |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment