Also CopyFile() doesn't preserve sparseness.

So, don't use UTF-8 literals when trying to represent invalid byte sequences. Just use standard string literals.

I wrote these while working on Windows support but they apply to Unix too.

I started from a clone of the Unix version of filesystem-disk.c++, and modified from there. The changes are pretty invasive, so I don't think trying to share implementations would have worked well.

Mostly adding / changing comments. Also some tweaks to consider that Windows uses HANDLEs rather than file descriptors.

Just a random thing I noticed.

Windows support will be implemented in a separate file.

There are two purposes:

1. Convert all the way to/from wchar_t, to make interfacing with Win32 filesystem APIs easier.
2. Use and handle '\\?\'-style paths. This prefix apparently opts into support for longer filenames. This seems like a good idea for any new programs! (It also opts out of support for using forward slashes as path separators and handling of '..', but kj::Path already handles these, so great.)

A parameter type of `String` takes precedence over `StringPtr` when matching a parameter of type `String&` (i.e. an lvalue), but then doesn't work since `String` has no copy constructor. Matching `String&&` instead avoids this problem.

Especially, this adds KJ_WIN32_HANDLE_ERRORS() to parallel KJ_SYSCALL_HANDLE_ERRORS().

Also include the error number in the string even if there is a message, because otherwise looking them up is hard.

Also remove the trailing newline from the message.

Different platforms have different sizes for wchar_t. For example:

* Linux: 32-bit (originally intended as UCS-4, rarely used in practice)
* Windows: 16-bit (originally intended as UCS-2, but now probably treated as UTF-16)
* BeOS: 8-bit (strictly intended to be UTF-8)

For KJ purposes, we'll assume wchar_t arrays use the UTF encoding appropriate to their size, whatever that may be on the target platform.

This is mainly being added because the Win32 API uses wchar_t heavily.

This allows arbitrary char16 arrays to round-trip through UTF-8 without losing information, even if the char16 arrays are not valid UTF-16.

This is necessary e.g. for filesystem manipulation on Windows, where filenames contain 16-bit characters but valid UTF-16 is not enforced.

Invalid UTF-16 represented in UTF-8 is affectionately known as WTF-8: http://simonsapin.github.io/wtf-8/

timer.h takes the place of the old time.h, while time.h itself has been pared down to only non-async-related stuff.

Since async-io.h includes timer.h, I suspect no one will be broken by this change.

I could have instead moved the non-async parts of time.h into a new header to avoid any possibility of breakage, but time.h is the correct name for the core header, and timer.h is the correct name for the async parts (the Timer class)... so I'm risking it.

(Later, we should add a nice subprocess API around this, maybe even one that can work on Windows too...)

This should prevent stack traces from spurriously pointing at the line after the one where the call actually happened.

This leaks due to the old compiler bug:

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=33799
http://llvm.org/bugs/show_bug.cgi?id=12286

Previously we only closed immediately after a response finished.

Fixes #286.

I intended to do this, like, 4 years ago.

It is invalid to pass null as a pointer argument to memcpy/memcmp/memset, even if the count argument is zero:

> Where an argument declared as size_t n specifies the length of the array for a
function, n can have the value zero on a call to that function. Unless explicitly stated
otherwise in the description of a particular function in this subclause, pointer arguments
on such a call shall still have valid values [...]

Detected by -fsanitize=nonnull-attribute.

If we finish decoding in step_a state, there is no current output character, so reading *plainchar will either be an uninitialized read or (if the output buffer is minimally sized) a past-the-end read.

Detected by -fsanitize=address.