twinBASIC Update: December 5, 2021
Highlights include the implementation of additional File System features, improved performance for Split and Replace, and a discussion about universal binaries.
On April 23, 2021, I helped Wayne Phillips introduce the world to twinBASIC at the Access DevCon Vienna conference. I boldly predicted that twinBASIC (along with the Monaco editor) would replace VBA and its outdated development environment by 2025. With that goal in mind, this weekly update is my attempt to keep the project fresh in the minds of the VBA development community.
Every Sunday, I will be providing updates on the status of the project, linking to new articles discussing twinBASIC, and generally trying to increase engagement with the project. If you come across items that should be included here, tweet me @NoLongerSet or email me at mike at nolongerset dot com.
Additional File System Features Implemented
Last week saw the first big push for the File System features of VBx, both in terms of language statements and standard library methods. Here's the full list of implemented features with newly implemented features from this week in bold:
Language Statements Implemented
- Open (Binary / Random modes only)
VBA.FileSystem Library Members Implemented
File System Features Not Yet Implemented
According to Wayne, the following statements have not yet been implemented (though given Wayne's pace of development, I expect this list will be outdated quickly):
- Input #
- Line Input #
String Function Performance Improvements
GitHub user KarlSevenSeven has been doing some performance testing of various twinBASIC string functions, including:
KarlSevenSeven's initial results showed that tB was much slower than the VBx implementations of these functions. Wayne went straight to work, making a few tweaks to the internal tB algorithm so that Split performed "somewhat better" and Replace was "somewhat improved."
In true British fashion, Wayne downplayed the actual performance gain.
The Split implementation improved by a factor of 100, resulting in a 30x improvement over the VBx approach (at least for KarlSevenSeven's large string benchmark).
The Replace implementation improved by a factor of 1,000 (!), resulting in a function that performs hundreds of times better (200x - 400x) than VBx when working with large strings.
Amused at Wayne's understatement, Ben Clothier posted:
I'm afraid to see what'd happen if you actually put real efforts into improving it. 😆
To which Wayne responded:
That's the exciting part... when I get more time to look at performance, things are going to get really interesting!
We're barely scratching the surface at the moment.
Around the Web
Ben Clothier started a discussion around how twinBASIC might support deployment to end users that may be running either 32-bit VBA or 64-bit VBA:
It would be much more convenient to be able to ship a DLL with a Office application that would run in either mode [(32-bit or 64-bit)]. While one can create an installer which can then install the proper version, nothing can beat the simplicity of a xcopy development.
Therefore, my wish is that it would be possible to build an output that has both 32-bit and 64-bit binaries embedded and ship with some kind of bootstrapper so that when it is used, the correct bitness will be used.
Wayne went on to discuss how .NET accomplishes this trick via the "Any CPU" compilation option.
As I understand, this special support for the CLR loading is baked into the OS, and is exclusively for .NET executables.
In other words, there's probably no way to get exactly what we want here.
That is, unless Wayne eventually makes his borderline-magical Virtual-COM feature available via twinBASIC:
This was one of the main pain points that my Virtual-COM technology solves. Virtual-COM, as seen in vbWatchdog, integrates the nativecode binaries into the host VBA project, meaning there are no DLLs needing to be distributed to end-users at all and the correct nativecode bitness is picked at runtime automatically. FWIW, I'd love to see Virtual-COM become integrated into twinBASIC at some point, as it solves so many distribution problems.
How VBA Code Gets Stored
So this is a bit off-topic, but I found it so interesting I just had to include it here.
I have on my to-do list an article that discusses VBA "compilation," but I'm not an expert on it myself. Outside of the internal Microsoft VBA team, though, I'm not sure there's a better authority on this stuff than Wayne Phillips. Here's Wayne's brief explanation for how VBA code gets stored:
VBA code gets stored in three versions.
1. the raw source code (compressed)
2. the tokenized version of the source code that the IDE uses for live editing. This is practically a 1:1 version of the raw source code, but in a tokenized form.
3. the 'compiled' version that Microsoft internally calls ExCode. (This is the version that is stored in compiled ACCDE and MDE files, with the other two forms stripped out.)
Important to note is that the third form is NOT proper native-code. It is a form that runs through a virtual machine. In VB6 terms, the third form is equivalent to compiling an EXE with the 'pcode' option of the VB6 compiler. VBA never gets turned into fully compiled nativecode - it always runs as ExCode (or 'pcode' in VB6 terms).
Advent of Code 2021
Regular twinBASIC commenter FullValueRider is participating in the seventh annual Advent of Code:
Advent of Code is an Advent calendar of small programming puzzles for a variety of skill sets and skill levels that can be solved in any programming language you like.
Why is this noteworthy?
Because this year, FullValueRider is using twinBASIC to compete in the challenge:
For the past couple of years I've been using VBA. This year I'm looking forward to using the new and exciting VBA replacement - twinbasic.
You can follow along with the project on GitHub. The twinBASIC source files are available at this link.
Best of luck, FullValueRider!
Here are the updates from the past week. You can also find this information by installing the twinBASIC VS Code extension and clicking on the Changelog tab of the extension page:
[v0.13.19, 5th December 2021]
- improved: return value optimization, allowing for move semantics instead of copy semantics in many return value situations [ https://github.com/WaynePhillipsEA/twinbasic/issues/522 ]
[v0.13.18, 4th December 2021]
- added: InStr$() function as a more efficient version of InStr without wrapping in Variants
- improved: better performance of InStr()
- fixed: edge case in using Split(vbNullString) [ https://github.com/WaynePhillipsEA/twinbasic/issues/522 ]
[v0.13.17, 3rd December 2021]
- improved: better performance of Replace() - tweaked the algorithm to be more performant on larger strings [ https://github.com/WaynePhillipsEA/twinbasic/issues/524 ]
[v0.13.16, 3rd December 2021]
- fixed: functions in the VBA.String module that take a Compare parameter now correctly reflect the Option Compare mode in effect if the Compare argument is not supplied
- improved: further improvements to Split() for binary modes [ https://github.com/WaynePhillipsEA/twinbasic/issues/521 ]
[v0.13.15, 3rd December 2021]
- improved: better performance of Split() - tweaked the algorithm to be more performant on larger strings [ https://github.com/WaynePhillipsEA/twinbasic/issues/521 ]
[v0.13.14, 2nd December 2021]
- fixed: compiler crash on syntax error for constructors involving generics [ https://github.com/WaynePhillipsEA/twinbasic/issues/520 ]
[v0.13.13, 2nd December 2021]
- fixed: File I/O print buffer was not correctly being flushed before close file [ https://github.com/WaynePhillipsEA/twinbasic/issues/279#issuecomment-984123084 ]
- fixed: File I/O relative paths support broken since v0.13.8 [ https://github.com/WaynePhillipsEA/twinbasic/issues/279#issuecomment-984123084 ]
[v0.13.12, 1st December 2021]
- fixed: deadlock when creating COM class instances on x64 in some instances [ https://github.com/WaynePhillipsEA/twinbasic/issues/516 ]
[v0.13.11, 1st December 2021]
- fixed: .twin file UTF8-BOM parser issue causing random failing parsing in some instances [ https://github.com/WaynePhillipsEA/twinbasic/issues/514 ]
[v0.13.10, 1st December 2021]
- fixed: missing compilation error for late-bound calls where a named-argument is followed by non-named arguments [ https://github.com/WaynePhillipsEA/twinbasic/issues/510 ]
- fixed: ReDim was causing most of the compiler to be linked in to final executables, causing bloatage and other linker side effects [ https://github.com/WaynePhillipsEA/twinbasic/issues/514 ]
- fixed: Linker hard crash [ [ https://github.com/WaynePhillipsEA/twinbasic/issues/514 ]
- fixed: sometimes optional default variant values would cause type library generation to fail [ https://github.com/WaynePhillipsEA/twinbasic/issues/512 ]
[v0.13.9, 29th November 2021]
- added Lock # and Unlock # statements for File I/O [ https://github.com/WaynePhillipsEA/twinbasic/issues/279 ]
- improved: compiler error information corrected for 'named arguments are not allowed in calls to procedures that have a ParamArray parameter' [ https://github.com/WaynePhillipsEA/twinbasic/issues/510 ]
[v0.13.8, 29th November 2021]
- added: Write syntax now supported in File I/O [ https://github.com/WaynePhillipsEA/twinbasic/issues/279 ]
- added: VBA.FileSystem.MkDir and VBA.FileSystem.RmDir implementations [ https://github.com/WaynePhillipsEA/twinbasic/issues/279 ]
- fixed: potential off-by-one error when using Tab() in Print statements [ https://github.com/WaynePhillipsEA/twinbasic/issues/279 ]
- improved: file I/O can now handle long file paths [ https://github.com/WaynePhillipsEA/twinbasic/issues/279 ]