Long, long, long paths
You may not be familiar with the MAX_PATH limitation in Windows, but if you are then you know how much of a nuisance it can be. Windows, unlike Unix, limits the full path of a file to 260 characters. People of a certain vintage who remember the 8.3 file name format will laugh at 260 characters being called a limitation but to Unix followers the notion of such a limit is crazy.
That 260 character limit is there, has been for a very long time, and is even present in the current incarnation: Windows 10.
Why is this a problem? Normally it is not and one can work away for years and not even be aware of it, but as luck would have it I hit that limit earlier today. You see, I was examining a Node.js project and it is quite normal for dependencies to be stored in a sub-directory (i.e. sub-folder) called node_modules, which in turn can have their own dependencies (in sub-sub-directories) and so on. These dependencies can run quite deep.
I had not created this project structure myself, but was examining it within a Zip file with the help of 7Zip. Then I decided to extract the entire hierarchy so that I could apply some tools to the collection. The extraction seemed to go OK, the tool extracted the entire collection into a temporary folder and then attempted to move it to its final destination. At 90% of the way an error message popped up, saying:
Destination Path Too Long
The file name(s) would be too long for the destination folder. You can
shorten the file name and try again, or try a location that has a shorter
path.
Nasty. Fortunately there is a little-known feature in Windows 10. Run the Group Policy Editor (gpedit.msc) and navigate to:
Local Computer Policy | Computer Configuration | Administrative Templates | System | Filesystem*
There you should find an entry called “Enable Win32 Long Paths”, which you should enable. This allows many Win32 programs, developed when MAX_PATH was 260 characters, to now create longer paths.
I made the change and rebooted. (That was probably unnecessary, but after a GP change it’s probably a good idea.) The result? Another failure. Once again the error message appeared when 7Zip was moving the extracted files to their final destination. This operation appears to be done by Windows itself, given the appearance of the dialog box that summarizes the progress of files during the move.
Although rather annoyed by this failure, I tried one more approach. Instead of dragging the root folder out of the 7Zip GUI to the target folder, I used the context menu (right-click) to cause an “extract here”. This bypasses the extraction to a temporary folder and dumps the extraction piece by piece into the target. This time it worked.
Out of curiosity, I then dragged the root of the newly extracted hierarchy to the same window, thus initiating a complete copy of the hierarchy in-situ using only Windows’ built-in facilities. It failed at exactly the same place.
What did I learn from this? It would appear that although the “Enable Win32 Long Paths” can be enabled in Windows 10, and tools like 7Zip appear to work with long file names, it seems that the change does not do anything for Windows Explorer as its own copy operation fails because of the old MAX_PATH limit. Where there’s one problem, others are sure to be found.
This is probably going to make my ongoing work in Node on Windows a little painful. Maybe I should stick to Unix.
* On some older versions of Win10 you may have to drill a step further down to “NTFS”.
PS In a post on the npm Blog two years ago they mentioned that npm3 was going to have flatter node_modules dependency hierarchies. Not a complete fix for the Windows limit, but a lot better than the bottomless pit that triggered the MAX_PATH issue. This doesn’t help me, however, because I’m trawling through legacy code that started life before the flatter approach appeared.
Categorised as: Operating Systems