ÓÐûÓа취¿ÉÒÔÖªµÀÒ»´®×Ö·û´®ÊÇÓÃʲô±àÂëµÄ£¬»òÕßÖ±½Óת³ÉunicodeµÄ·½·¨Ò²¿ÉÒÔ¡£
-------------- 下一部分 --------------
Ò»¸öHTML¸½¼þ±»ÒƳý...
URL: http://python.cn/pipermail/python-chinese/attachments/20071030/fb277bd6/attachment.htm 

应当不行。
你写一堆的try，直到成功转成unicode。

在 07-10-30，Kevin anew<kevinanew在gmail.com> 写道：
> 有没有办法可以知道一串字符串是用什么编码的，或者直接转成unicode的方法也可以。
> _______________________________________________
> python-chinese
> Post: send python-chinese在lists.python.cn
> Subscribe: send subscribe to
> python-chinese-request在lists.python.cn
> Unsubscribe: send unsubscribe to
> python-chinese-request在lists.python.cn
> Detail Info:
> http://python.cn/mailman/listinfo/python-chinese
>


-- 
Blog http://vicalloy.spaces.live.com/
My googlepage http://vicalloy.googlepages.com/
OldPhoto http://www.lzpian.com/

猜吧:)
http://chardet.feedparser.org/ 这个不知道是不是你想要的

在07-10-30，Kevin anew <kevinanew at gmail.com> 写道：
>
> 有没有办法可以知道一串字符串是用什么编码的，或者直接转成unicode的方法也可以。
> _______________________________________________
> python-chinese
> Post: send python-chinese at lists.python.cn
> Subscribe: send subscribe to python-chinese-request at lists.python.cn
> Unsubscribe: send unsubscribe to  python-chinese-request at lists.python.cn
> Detail Info: http://python.cn/mailman/listinfo/python-chinese
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://python.cn/pipermail/python-chinese/attachments/20071030/4db82db2/attachment.htm 

贴了一百遍的东西

def zh2unicode(stri):
        """Auto converter encodings to unicode

        It will test utf8,gbk,big5,jp,kr to converter"""
        for c in ('utf-8', 'gbk', 'big5', 'jp', 'euc_kr','utf16','utf32'):
                encc = c
                try:
                        return stri.decode(c)
                except:
                        pass
        return stri



对比较短的文本,是无论如何都不能保证 100% 准确判断的,
mozilla 那套看字符出现概率的亦然.

¿ÉÒÔ²Â,µ«ÊÇû°ì·¨¿Ï¶¨,ÓÈÆäÊǶÔgbkºÍbig5.
±ÈÈç:
>>> print '±ä¾Þ'.decode('big5')
²Ü²Ù
>>> print '±ä¾Þ'.decode('gbk')
±ä¾Þ


ÔÚ07-10-30£¬Kevin anew <kevinanew在gmail.com> дµÀ£º
>
> ÓÐûÓа취¿ÉÒÔÖªµÀÒ»´®×Ö·û´®ÊÇÓÃʲô±àÂëµÄ£¬»òÕßÖ±½Óת³ÉunicodeµÄ·½·¨Ò²¿ÉÒÔ¡£
> _______________________________________________
> python-chinese
> Post: send python-chinese在lists.python.cn
> Subscribe: send subscribe to python-chinese-request在lists.python.cn
> Unsubscribe: send unsubscribe to  python-chinese-request在lists.python.cn
> Detail Info: http://python.cn/mailman/listinfo/python-chinese
>
-------------- 下一部分 --------------
Ò»¸öHTML¸½¼þ±»ÒƳý...
URL: http://python.cn/pipermail/python-chinese/attachments/20071030/0be260ab/attachment.html 

月经贴
chardet

在07-10-30，Samuel Chi <princeofdatamining at gmail.com> 写道：
>
> 可以猜,但是没办法肯定,尤其是对gbk和big5.
> 比如:
> >>> print '变巨'.decode('big5')
> 曹操
> >>> print '变巨'.decode('gbk')
> 变巨
>
>
> 在07-10-30， Kevin anew <kevinanew at gmail.com> 写道：
> >
> > 有没有办法可以知道一串字符串是用什么编码的，或者直接转成unicode的方法也可以。
> > _______________________________________________
> > python-chinese
> > Post: send python-chinese at lists.python.cn
> > Subscribe: send subscribe to python-chinese-request at lists.python.cn
> > Unsubscribe: send unsubscribe to
> > python-chinese-request at lists.python.cn
> > Detail Info: http://python.cn/mailman/listinfo/python-chinese
> >
>
>
> _______________________________________________
> python-chinese
> Post: send python-chinese at lists.python.cn
> Subscribe: send subscribe to python-chinese-request at lists.python.cn
> Unsubscribe: send unsubscribe to  python-chinese-request at lists.python.cn
> Detail Info: http://python.cn/mailman/listinfo/python-chinese
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://python.cn/pipermail/python-chinese/attachments/20071030/b8215aa7/attachment-0001.htm 

"mozilla ÄÇÌ׿´×Ö·û³öÏÖ¸ÅÂʵÄÒàÈ»." ÄÇÀïÓÐÏÂÔØÂð?Ïë¿´¿´ËûÊÇÔõôÊÇ×öµÃ

ÔÚ07-10-30£¬Jiahua Huang <jhuangjiahua在gmail.com> дµÀ£º
>
> ÌùÁËÒ»°Ù±éµÄ¶«Î÷
>
> def zh2unicode(stri):
>         """Auto converter encodings to unicode
>
>         It will test utf8,gbk,big5,jp,kr to converter"""
>         for c in ('utf-8', 'gbk', 'big5', 'jp', 'euc_kr','utf16','utf32'):
>                 encc = c
>                 try:
>                         return stri.decode(c)
>                 except:
>                         pass
>         return stri
>
>
>
> ¶Ô±È½Ï¶ÌµÄÎı¾,ÊÇÎÞÂÛÈçºÎ¶¼²»Äܱ£Ö¤ 100% ׼ȷÅжϵÄ,
> mozilla ÄÇÌ׿´×Ö·û³öÏÖ¸ÅÂʵÄÒàÈ».
> _______________________________________________
> python-chinese
> Post: send python-chinese在lists.python.cn
> Subscribe: send subscribe to python-chinese-request在lists.python.cn
> Unsubscribe: send unsubscribe to  python-chinese-request在lists.python.cn
> Detail Info: http://python.cn/mailman/listinfo/python-chinese
-------------- 下一部分 --------------
Ò»¸öHTML¸½¼þ±»ÒƳý...
URL: http://python.cn/pipermail/python-chinese/attachments/20071030/cc8f68fa/attachment.html 

ͬÇóÕâ¶Î´úÂ룬£ç£ï£ï£ç£ì£åÁËÒ»ÏÂûÕÒµ½


2007-10-30 



ljdamjd 


 
"mozilla ÄÇÌ׿´×Ö·û³öÏÖ¸ÅÂʵÄÒàÈ»." ÄÇÀïÓÐÏÂÔØÂð?Ïë¿´¿´ËûÊÇÔõôÊÇ×öµÃ
-------------- 下一部分 --------------
Ò»¸öHTML¸½¼þ±»ÒƳý...
URL: http://python.cn/pipermail/python-chinese/attachments/20071030/4fbc2a0a/attachment.htm 

ÎÒ²âÊÔÁËÒ»ÏÂchardet, µ«·¢ÏÖËû°ÑÓз±ÌåÖÐÎĵÄÄÚÈݺÃÏñÒ²ÅÐ¶Ï³É gb2312 ÁË. ÐǪ̈ʹÓÃʱÓÐûÓз¢ÏÖÕâ¸öÎÊÌâ? ÄãÊÇÈçºÎ½â¾öµÄÄØ?

ÔÚ07-10-30£¬3751 <lwm3751在gmail.com> дµÀ£º
>
> Ô¾­Ìù
> chardet
>
> ÔÚ07-10-30£¬Samuel Chi <princeofdatamining在gmail.com> дµÀ£º
> >
> > ¿ÉÒÔ²Â,µ«ÊÇû°ì·¨¿Ï¶¨,ÓÈÆäÊǶÔgbkºÍbig5.
> > ±ÈÈç:
> > >>> print '±ä¾Þ'.decode('big5')
> > ²Ü²Ù
> > >>> print '±ä¾Þ'.decode('gbk')
> > ±ä¾Þ
> >
> >
> > ÔÚ07-10-30£¬ Kevin anew <kevinanew在gmail.com> дµÀ£º
> > >
> > > ÓÐûÓа취¿ÉÒÔÖªµÀÒ»´®×Ö·û´®ÊÇÓÃʲô±àÂëµÄ£¬»òÕßÖ±½Óת³ÉunicodeµÄ·½·¨Ò²¿ÉÒÔ¡£
> > > _______________________________________________
> > > python-chinese
> > > Post: send python-chinese在lists.python.cn
> > > Subscribe: send subscribe to python-chinese-request在lists.python.cn
> > > Unsubscribe: send unsubscribe to
> > > python-chinese-request在lists.python.cn
> > > Detail Info: http://python.cn/mailman/listinfo/python-chinese
> > >
> >
> >
> > _______________________________________________
> > python-chinese
> > Post: send python-chinese在lists.python.cn
> > Subscribe: send subscribe to python-chinese-request在lists.python.cn
> > Unsubscribe: send unsubscribe to
> > python-chinese-request在lists.python.cn
> > Detail Info: http://python.cn/mailman/listinfo/python-chinese
> >
>
>
> _______________________________________________
> python-chinese
> Post: send python-chinese在lists.python.cn
> Subscribe: send subscribe to python-chinese-request在lists.python.cn
> Unsubscribe: send unsubscribe to  python-chinese-request在lists.python.cn
> Detail Info: http://python.cn/mailman/listinfo/python-chinese
>
-------------- 下一部分 --------------
Ò»¸öHTML¸½¼þ±»ÒƳý...
URL: http://python.cn/pipermail/python-chinese/attachments/20071030/ee6a6fe9/attachment.html 

1. mozilla 和 MS 的所谓 gb2312 其实是 gb18030
2. 谁说繁体中文是 big5?
utf8 和 gb18030 包括了简繁体中文

在 07-10-30，Kevin anew<kevinanew at gmail.com> 写道：
> 我测试了一下chardet, 但发现他把有繁体中文的内容好像也判断成 gb2312 了. 兄台使用时有没有发现这个问题? 你是如何解决的呢?
>