At the moment SMBJ takes the UTF-16 byte sequence representing the file name from the message and attempts to decode it to a Java String. Then when you try to open the file it reencodes the Java String to a UTF-16 byte sequence again. In situations where the original input data is invalid UTF-16 you're going to get the replacement character code point and as a consequence the reencoded version may not be identical to the original. In other words, the UTF-16 -> String transformation is not guaranteed to be reversible.

The only way I can come up with to work around this is to hold on to the original 'raw' filename so that you can send it back to the server verbatim.

To actually answer your question, this is kind of a bug when dealing with filenames that are not valid UTF-16. There's no workaround for this. Proper fix would require changing the internal representation of paths in SMBJ from String to something lower level that holds on to the original byte representation of the filename.

@pepijnve , I appreciate your prompt reply. It sounds like there is no roundtrip conversion from UTF-16 to String and this explains the behavior.

The round-trip works when all characters are valid UTF-16. In your case one of them isn't.

@hierynomus yes, that's the case.

Just checked the MS-SMB2 spec:

Unless otherwise specified, all textual strings MUST be in Unicode version 5.0 format, as specified in
[UNICODE], using the 16-bit Unicode Transformation Format (UTF-16) form of the encoding. Textual
strings with separate fields identifying the length of the string MUST NOT be null-terminated unless
otherwise specified.

And for the SMB2 CREATE request:

Buffer (variable): A variable-length buffer that contains the Unicode file name ...

This means that the filename must be valid UTF-16 in order to work. It seems that the server is not adhering to the spec.

@mpohajac The sequence you provided, 323120EFBFBE61 is actually UTF-8 and decodes to the codepoint sequence 50 49 32 65534 97 or 2, 1, , �, a. 65534 is actually a valid codepoint and should roundtrip just fine. The alternative EFBFBD is 65533 and is also valid and should also roundtrip identically.

Would it be possible to provide a hexdump of the UTF-16 byte sequence that actually gets sent from the server instead? Wireshark, tcpdump or netsh packet capture would be ideal, but a hexdump from the Java code works too of course.

@hierynomus , @pepijnve thanks for the info.

These are the bytes from the wireshark for the Filename:

addr
01c0   32 00 31 00 20 00 fe ff 61 00                     2.1. ...a.

Four valid UTF-16 chars '2','1','<space>','a' and the one feff before 'a' that appears to be a sort of bom, but strange to appear in between.

This is reproducible with:

• the attached directory.zip, the filename inside has this character
• demo code

@hierynomus how would you prefer to handle this case? Best option IMO is to use something like the SmbPath class everywhere that can wrap a raw byte[]. In other words, defer conversion to Java String until it's requested. That would allow you to take the path object from a directory info and pass that on to open unprocessed.

I agree with you that what the server is sending is not 100% spec compliant, but that's just a fact of life we have to deal with. We see this kind of mess all the time, especially when data is written via NFS and then accessed via SMB.