[Bug] reading pcap files on non UTF-8 Windows system fails due to convert path to str fails #383

Activity

1. Stargateur commented on Mar 22, 2025

   

   Stargateur

   on Mar 22, 2025

   Last edited by Stargateur

   Contributor

   More actions

   I'm not sure I can fix it, pcap lib is design around unix ecosystem, https://www.tcpdump.org/manpages/pcap_open_offline.3pcap.html, the function is expected an char * and that always expected to be utf-8, I can probably manage to try to send a char * that is utf 16 encoded with a final nul bytes but I doubt it will work.

   Another way would be to use FILE * but it's end with the same problem, fopen expect an char *, one would need to use _wfopen or use other windows API. I believe it's too much work for one user and at this point use something dedicated for windows.

   So in conclusion... set your windows to UTF-8 or rename your file using ascii character.

   PS:

   The purpose of PathBuf & Path is to avoid encoding issue, to_str() is not an elegant way.

   If only this will be this simple...
2. Stargateur commented on Mar 22, 2025

   

   Stargateur

   on Mar 22, 2025

   Contributor

   More actions

   Please try #384 with utf-16 namefile
3. JackLiar commented on Mar 23, 2025

   

   JackLiar

   on Mar 23, 2025

   Author

   More actions

   Well, the commit doesn't seem to work on windows, the error is throw out is: failed to open C:\test\测试.pcapng, libpcap error: C: No such file or directory

   Windows sometimes don't use UTF-16 as default for Asian language users，for example GBK for Chinese, Shift_JIS for Japanese, EUC-KR for Korean, assuming UTF-16 on windows is not quite appropriate

   I'd like to help improve this part, I'd spent some time next week and see what we can cook :)
4. JackLiar commented on Mar 23, 2025

   

   JackLiar

   on Mar 23, 2025

   Last edited by JackLiar

   Author

   More actions

   I found the possible solution.

   There is a pcap_init function in wpcap/libpcap implemention, this is what it saids:

   /*
    * Do whatever initialization is needed for libpcap.
    *
    * The argument specifies whether we use the local code page or UTF-8
    * for strings; on UN*X, we just assume UTF-8 in places where the encoding
    * would matter, whereas, on Windows, we use the local code page for
    * PCAP_CHAR_ENC_LOCAL and UTF-8 for PCAP_CHAR_ENC_UTF_8.
    *
    * On Windows, we also disable the hack in pcap_create() to deal with
    * being handed UTF-16 strings, because if the user calls this they're
    * explicitly declaring that they will either be passing local code
    * page strings or UTF-8 strings, so we don't need to allow UTF-16LE
    * strings to be passed.  For good measure, on Windows *and* UN*X,
    * we disable pcap_lookupdev(), to prevent anybody from even
    * *trying* to pass the result of pcap_lookupdev() - which might be
    * UTF-16LE on Windows, for ugly compatibility reasons - to pcap_create()
    * or pcap_open_live() or pcap_open().
    *
    * Returns 0 on success, -1 on error.
    */
   
   int pcap_new_api;		/* pcap_lookupdev() always fails */
   int pcap_utf_8_mode;		/* Strings should be in UTF-8. */
   
   int
   pcap_init(unsigned int opts, char *errbuf)

   And I do some test about std::env::args_os, it returns a OsString Iterator, and the under lying encoding is UTF-8 not UTF-16 nor GBK nor other encodings(only test on my PC). I guess there is some deep Rust std library magic down there.

   So I think there could be two solutions:

   Expose pcap_init function to Rust users

   This way does not make any assumption about what user would pass to the pcap_open_offline function, it's all user's duty to remember call pcap_init at the beginning.

   Call pcap_init on every call

   This way guarantees this function is called, user don't needs to know such stuff even exists. However, there might be some rare occasions that this is wrong, and would lead to another confusing error.

   Hope this would help :)
5. cicku commented on Aug 19, 2026

   

   cicku

   on Aug 19, 2026

   Contributor

   More actions

   #401 only adds pcap_init into the comment, I can draft an implementation after then.

   👍React with 👍1Reacted by Wojciech Kozlowski
6. Wojtek242

   closed this as completedin #403on Sep 7, 2026