I closed the videothumbnailsmaker.exe program. Then tried double clicking various .vtx files. Some of them opened, some didn't. The ones that didn't, just changed the mouse cursor to an hourglass briefly, before back to a regular pointer.
I think I figured out why. All files with spaces in them didn't work. So I did the following that I think fixed it:
Windows Explorer, Tools, Folder Options, File Types.
Scroll down to VTX extension, Advanced.
Actions "open", Edit.
'application to perform actions' changed from 'c:\path\to\VideoThumbnailsViewer %1' to 'c:\path\to\VideoThumbnailsViewer "%1" ' (enclose %1 in double quotes).
The vtx filename was probably getting passed to the viewer as multiple commandline options. The viewer is probably only checking the first option (aka $1 in .bat files). Enclosing the file (%1) in double quotes should ensure the entire filename, spaces included is the only option passed.
Some of my videos have all blank thumbnails, but I haven't played much with the options, besides enabling the extreme engine..