twinBASIC Update: January 28, 2024

Mike Wolfe -

On April 23, 2021, I helped Wayne Phillips introduce the world to twinBASIC at the Access DevCon Vienna conference.  I boldly predicted that twinBASIC (along with the Monaco editor) would replace VBA and its outdated development environment by 2025.  With that goal in mind, this weekly update is my attempt to keep the project fresh in the minds of the VBA development community.

Every Sunday, I will be providing updates on the status of the project, linking to new articles discussing twinBASIC, and generally trying to increase engagement with the project.  If you come across items that should be included here, tweet me @NoLongerSet or email me at mike at nolongerset dot com.

Here are some links to get involved with the project:

Highlights

The New IDE is Here!

After a 73-day hiatus from near-daily beta releases–including a week of radio silence that had some twinBASIC supporters checking the UK's missing persons website–Wayne emerged from his laboratory with BETA 424, the long-awaited release of the redesigned twinBASIC IDE.

The new release was way more than just a new IDE, though.  There were also several ambitious code-level improvements, including:

  • Support for static linking of C/C++ libraries
  • Experimental type inference support
  • Avoidance of Variant coercion for the If() ternary operator
  • Codegen improvements resulting in 20-30% smaller executables

While the scale of the new release has inevitably led to several regressions, Wayne has since released a flurry of bug fixes, signaling a return to his previous release tempo.

Customizable IDE Panel Layouts

One of the most anticipated features in the new IDE is the ability to rearrange the various panels of the IDE, including the:

  • Editor
  • Project Explorer
  • Open Editors
  • History
  • Outline
  • Call Stack
  • Variables
  • Watches
  • Debug Console
  • Problems
  • Package Publishing
  • Toolbar
  • Toolbox
  • Properties

The IDE comes with a couple of built-in layouts, the Default layout and the Full Screen Editor layout.  

The Default layout includes the following visible panels:

  • Toolbar
  • Toolbox
  • Editor
  • Debug Console
  • Problems
  • Call Stack
  • Variables
  • Project Explorer
  • Properties
The Default Built-In Layout

The Full Screen Editor layout is a more focused view that provides maximum screen real estate to the code editor.  It hides most of the panels except for the following:

  • Toolbar
  • Toolbox
  • Editor
  • Properties
The Full Screen Editor Layout

Given all the customizability available, another really nice feature of the IDE is the ability to save, share, and load custom layouts.  The Window menu item includes a Panel Layouts sub menu as seen here:

The Manage Panel Layouts screen is a dialog window that allows you to copy the JSON-based panel configuration for any of your saved layouts.  You can then share your layouts with other developers or transfer them to other development machines.  

The Manage Panel Layouts dialog window

The [Add New...] button appears to make a copy of the "<DEFAULT> (built-in)" panel layout's JSON, which you can easily paste over with configuration text from another developer or development machine.

Keyboard Shortcut Customizations

Tip from Wayne:

For anyone wanting to play with the keyboard shortcuts customization, if you want to know the commandID of something, you'll see the commandID listed in the bottom right corner of the status bar when you hover over a menu item or any button in the IDE.   In future of course these will be available directly in the shortcut editor, and will all be documented.

Static Linking

Besides the improved IDE, one of the key new features introduced in BETA 424 was support for using C static libraries (LIB/OBJ files) via the new Import Library syntax.

Here's some background information on static linking and how it works with twinBASIC (via Claude-2-100k on poe.com):

Static linking is a method of bundling external code into a program's executable file rather than relying on external DLL dependencies at runtime.

Some key points about static linking in twinBASIC:

  • Starting in beta 424, twinBASIC gained the ability to statically link C/C++ code compiled into .obj files.
  • This works by importing the .obj files into the project's Miscellaneous folder.
  • An example was shown using SQLite - the final EXE contained the full SQLite code embedded, removing the need for the sqlite3.dll dependency.
  • The size tradeoff is worth it for some cases to avoid DLL issues. For SQLite it added ~800KB.
  • Guidance has been provided on compiling C/C++ code into compatible .obj files that twinBASIC can consume.
  • Both .obj and .lib files can be used. The latter requires lib.exe when compiling.
  • Community member fafalone demonstrated static linking of the tinf compression library.
  • Statically linking small utilities like tinf removes dependencies and makes sharing projects easier.
  • There are still some quirks and limitations around static linking to be improved in future releases.

Overall, static linking opens new possibilities for integrating external C/C++ code into self-contained twinBASIC programs without runtime DLLs. This helps avoid dependency issues. As the capabilities mature, static linking will become an important tool for developing complex twinBASIC applications.

The sample referenced in the above output is Sample 17. Static Library Example (SQLITE3):

Here are Wayne's introductory code comments from the MainModule in Sample 17:


' SQLITE3 static library example
' With this example, the generated executable does not require 
'  any external DLL files as the SQLITE3 code is linked statically
'  into the executable.


' This project contains the compiled OBJ files from the SQLITE3 project, 
'   compiled with MSVC
' Please review the SQLITE3 licence:  https://www.sqlite.org/copyright.html
' ---------------------------------------------------------------------------
' HOW TO COMPILE SQLITE3 STATIC LIBRARY YOURSELF:
'   USE THE AMALGATION VERSION OF SQLITE (contained in this project)
'
'   Open the MSVC Developer Prompt 
'      (MSVC > Tools menu > Command Line > Developer Command Prompt)
'
'       To compile 32-bit OBJ files:
'       enter:    
'           vcvars32.bat
'           cl /c /Gw /Gy /GS- /DSQLITE_OMIT_SEH sqlite3.c /Fo"sqlite3_32.obj"
'
'       To compile 64-bit OBJ files:
'       enter:    
'           vcvars64.bat
'           cl /c /Gw /Gy /GS- /DSQLITE_OMIT_SEH sqlite3.c /Fo"sqlite3_64.obj"
'
'   Afer compilation, import the sqlite3_32.obj and sqlite3_64.obj files
'      into the Miscellaneous folder in this project
' ---------------------------------------------------------------------------


#If Win64 Then
    Import Library "/Miscellaneous/sqlite3_64.obj" As SQLITE3 Link "stdlib", "kernel32"
#Else
    Import Library "/Miscellaneous/sqlite3_32.obj" As SQLITE3 Link "stdlib", "kernel32"
#End If

Discord Chat Summary

* Auto-generated via Claude-2-100k on poe.com

Here is a summary of the key points from the General channel Discord chat transcript on the development of twinBASIC over the past week:

Introduction

The transcript covers conversations in the twinBASIC Discord server's #general channel from January 20th to January 28th, 2024. This was an exciting week for the project, with the release of major IDE updates in betas 424-430.

Key Points

  • Beta 424 released with a completely redesigned IDE UI, including docking, theming, and other improvements. Some instability and bugs reported.

  • Beta 425 enabled early static library support through linking .obj files. An example project using SQLite was provided.

  • Beta 426 aimed to improve stability but still had some crashing issues reported.

  • After a period of no updates, Wayne returned and gave insight into needing more time to stabilize features before releasing.

  • Beta 430 released soon after, with Wayne mentioning it should fix an internal error related to number of forms.

  • Discussions around best practices for compiling C/C++ code into .obj files that twinBASIC can use. Detailed guidance was provided.

  • Static linking of the tinf compression library demonstrated by community member fafalone.

  • Overall impressions of the new IDE were very positive, despite some bugs. The update is seen as a major step forward.

Conclusion

The week saw significant progress on delivering a more modern IDE workflow to twinBASIC users. Static library support also opens up possibilities for integrating C code without dependencies on DLLs. There is still instability and crashes being worked through, but the releases show twinBASIC continuing to mature towards v1.0.

Here is a summary of the key points from the Docs channel Discord chat transcript on the development of twinBASIC over the past week:

  • Community member fafalone updated the New Features wiki page to document the changes introduced in beta 424. These include the major IDE redesign, theming, docking, project templates, and more.

  • The wiki documentation keeping pace with new twinBASIC releases is important for users to understand what's changed and how to leverage the new capabilities.

  • While only one message this week, it highlights the ongoing community effort to maintain documentation alongside twinBASIC's rapid evolution towards v1.0.

  • Good documentation will be key to onboarding new users once twinBASIC hits major milestones like v1.0. The community is laying the foundation for that now through the wiki.

Overall, the updates to the New Features wiki this week show the documentation keeping up with twinBASIC's progress. This work by the community will pay dividends when twinBASIC is ready for a wider audience. Robust documentation greases the wheels for adoption of any software project.

Around the Web

Interlocked64: A Demo of twinBASIC's New Static Linking Feature

Here's fafalone's announcement on the twinBASIC Discord show-and-tell channel:

A while back I made Interlocked64.dll. The Interlocked* APIs, which perform atomic operations important for multithreading and kernel mode, are exported by the Windows API only in 32bit; in 64bit, they're only available as compiler intrinsics. So I wrote a C++ DLL to export these intrinsics as callable functions. But you can't use such a DLL in kernel mode. tB now allows for static linking of .obj and .lib files, so I've made a version of this library for use with tB's static linking, which can be used both in regular programs and kernel drivers. It will be merged into the tbKMode package next time that's updated.

From an x64 developer tools prompt, or from a regular one after running vcvars64.bat, compile with: cl.exe /c /Gw /Gy /GS- Interlocked64.c /Fo"Interlocked64.obj"

Then import into the Miscellaneous folder of your project. After that, it's used like this:

#If Win64 Then
Import Library "/StaticLibraries/Interlocked64.obj" As Interlocked
#End If

Module MyModule
    
    Private DeclareWide PtrSafe Function InterlockedIncrement Lib Interlocked Alias "x64InterlockedIncrement" (Addend As Long) As Long
...

Since it's just a wrapper for compiler intrinsics, there's no dependencies to link with Link "module1", ... like the SQLITE example.

Interlocked64.c and Interlocked.obj are available in the Interlocked64.dll repository:
https://github.com/fafalone/Interlocked64

GitHub - fafalone/Interlocked64: x64 Interlocked* compiler instrinsics exported by standard DLL
x64 Interlocked* compiler instrinsics exported by standard DLL - GitHub - fafalone/Interlocked64: x64 Interlocked* compiler instrinsics exported by standard DLL
GitHubfafalone

A Brief Reminder that Wayne is Human, After All

Remember that you are messaging a human, not a robot. That's all!

The gentle chastisement above was delivered by someone (not Wayne) in the Discord chat this past week to a commenter who appeared to be losing his patience and getting a bit sarcastic when asking about the timeline for the latest update and the upcoming release of version 1 (to be fair, it wasn't entirely clear whether the original commenter's message was posted with malice, a playful tone that didn't come across as intended, or was a miscommunication due to English not being the commenter's first language).

Nevertheless, Wayne acknowledged that the recently updated roadmap may require further revisions.  I assume it's a mix of the underlying complexity of such a massive undertaking as twinBASIC, short-term financial realities (creating a new programming language is the ultimate long-term investment), and good old-fashioned burnout.

Following concern about his recent radio silence, Wayne assured everyone he just had his head down finishing the big update:

Thanks, and thank you to all for your well-wishes.  I just had to shut-off from here for a little while to concentrate on the update.  I feel like I haven't slept for two months

Another user replied he considered "nose to the grindstone" a possible explanation but that he had other "nagging worries."  Wayne assumed "those nagging worries" were references to the graveyard of vaporware that comprise previous attempts at building a VB6 successor (they were not, as it turns out).  Regardless, Wayne replied:

I know, and I can appreciate those feelings given the history of failed similar projects we've seen over the past 20 years or so.  But they weren't developed by me... and those that know me know that I don't give up

A couple hours later (after the aforementioned sarcastic comments about the roadmap), Wayne admitted the project has taken a toll on him but that he still remains fully committed to it:

Realistically, yes, the roadmap probably needs to be replanned.  The financial pressures are now biting a little.  Until now I've given pretty much 24/7 to this project, despite the project not providing enough income yet to live on.  Unfortunately this situation cannot continue indefinitely, and so going forward I will likely have to slightly reduce my time on tB a little to work on the other side of my business.

Importantly, note that my commitment to, and my belief in the future of twinBASIC has not changed at all, and delivering v1 is still in progress.

Reading the tea leaves here, I fully expect the release date for v1 to get pushed back until at least the end of 2024.  

That said, nothing I have seen over the past three years leads me to any other conclusion than Wayne will achieve a v1 release of twinBASIC.  

Personally, I don't think getting the IDE modernized for a v1 release was part of the initial plan.  However, I think over time Wayne came to see it as a more important requirement from a public relations standpoint.  twinBASIC is already swimming upstream against a programming community that will dismiss it as a legacy language due to its roots in VB6.  Launching with an IDE lacking in modern bells and whistles would have only cemented that view.

In other words, I think the IDE improvement was a bit of an unplanned task (especially given the scale of the upgrade) that played a major role in pushing back the v1 release.  I also think it was the right strategic decision for the long-term prospects of twinBASIC as a programming language.

Changelog

Here are the updates from the past week.  You can also find this information by visiting the GitHub twinBASIC Releases page.

Releases · WaynePhillipsEA/twinbasic
Contribute to WaynePhillipsEA/twinbasic development by creating an account on GitHub.
GitHubWaynePhillipsEA

ChatGPT Changelog Summary

* Auto-generated via ChatGPT, sorted in order of ChatGPT's opinion of "most impactful changes."

  • Warning: This is a preview release with known memory leaks, disabled Edit-And-Continue, ADDINS, and custom controls forms support, and potential IDE and debugger sync issues.
  • Major Changes:
    • Refactor: Significant refactor of parsing stages, leading to potential regressions.
    • Debugger: Various debugger stability improvements and a fix for a crash when hitting the Stop button on x64.
    • IDE Makeover: Complete overhaul of the IDE with more flexible dockable panels, improved editor, and customizable keyboard shortcuts (~300 commands).
  • New Features:
    • Type Inference: Experimental support for local variable type inference with 'As Any' syntax.
    • C Static Libraries: Support for using C static libraries (LIB/OBJ files) via new Import Library syntax.
    • Form Designer: Experimental feature for colored resize/movement grabbers and improved mouse capture.
  • Improvements:
    • Code Generation: Code size reductions of approximately 20-30% and removal of "procedure too large" limits.
    • Performance: Enhanced class field get/let/set auto implementations for better performance and smaller codesize, as well as internal optimizations of standard library functions.
    • IDE Enhancements: Improved dock panel visibility, resizable bar interaction, hover functionality in LOCAL/WATCH panels, and various UI/UX tweaks.
  • Bug Fixes:
    • Fixed various code generation issues, clipboard-paste cursor positioning in the IDE editor, and form Caption property acknowledgment at runtime.

WARNING: The following issues are present in BETA builds 424 - 430 (the latest build as of publication):

BETA 424

  • WARNING: This is a PREVIEW of the significant changes made over the last two months.  Regressions are EXPECTED at this stage.
  • WARNING: (temporary issue) there are currently some sync issues with regards to the IDE and internal debugger state
  • WARNING: (temporary issue) Edit-And-Continue support has been disabled in this build
  • WARNING: (temporary issue) ADDINS are disabled in this build
  • WARNING: (temporary issue) memory usage will likely be HIGHER in this build than previous builds
  • WARNING: (temporary issue) custom controls forms support currently disabled in this build
  • changed: significant refactor of parsing stages.  regressions likely in this build
  • removed: limits on "procedure too large" during codegen
  • fixed: edge case of VBA.Val implementation could cause a runtime crash
  • improved: all Global/App/Clipboard/Forms/Screen global object members are now independently linked-in as necessary, through the use of RedirectToStaticImplementation attribute
  • all sorts of codegen improvements and bug fixes (typical 20-30% code size reduction)
  • improved: some improvements to general stability
  • fixed: hitting the Stop debugger button whilst debugging at an explicit 'Stop' statement on x64 would crash-restart the compiler
  • added: experimental support for local variable type inference using temporary 'As Any' syntax (e.g. Dim something As Any = 5& gives a typed variable of type Long)
  • improved: If() operator function now works directly on all basic types, without coercing the values to Variants, provided both outcomes have matching types (e.g. If(cond,ALong,ALong) does not go through Variant, but If(cond,ALong,AInteger) does)
  • improved: some improvements to general Debugger stability
  • fixed: Debugger object reference leak while stepping through code in some instances
  • improved: completely reworked class field get/let/set auto implementations for better performance and smaller codesize
  • improved: IDE monaco editor now better handles hex/octal/binary literals, e.g. selecting the full &H12345678 text rather than just H12345678 when double clicking on it
  • improved: hover on LOCAL/WATCH panels, now show expanded view of the underlying variable, with clipboard copy ability
  • improved: the left/right/bottom dock panels can now be hidden individually (Window menu > Toggle {X} Dock Panel)
  • improved: resizable bars in the IDE now have a short delay before showing blue, giving a less jarring experiencing when moving the mouse over these areas
  • improved: IDE complete makeover, including more flexible dockable panels support
  • added: support for using C static libraries (LIB/OBJ files) from within twinBASIC, using the new Import Library syntax, see SQLITE3 Static Library sample
  • improved: internal optimizations of standard library functions such as memcpy, memcmp
  • improved: dragging the yellow execution arrow during a debug session now allows for the mouse pointer to move outside of the document area keeping the drag ability
  • improved: every IDE action is now available to be linked to a customizable keyboard shortcut (~300 commands currently)
  • improved: FORMAT IDE menu items are all now implemented, along with most other menu items
  • added: experimental feature for showing different coloured resize/movement grabbers for multi-selected controls in form designer
  • improved: mouse capture is now set correctly in the form designer when dragging resize/movement grabbers
  • improved: standardized the scrollbars and scrolling behaviour across all panels and monaco editors
  • improved: standardized the context menus and root menu behaviours across all panels and monaco editors
  • fixed: codegen issues involving overloaded constructors combined with field initializers
  • fixed: IDE editor clipboard-paste would fail to ensure the new cursor position was visible (in view)
  • improved: new inline runtime-error monaco widget that allows for 'Try Again', 'Ignore', 'Stop', and 'Search Online' buttons
  • improved: better handling of new-line text indentation and auto-block-completion by performing a single edit after hitting ENTER rather than two edits
  • improved: project Settings is now a modal dialog rather than a tabbed document, ensuring that Save/Cancel is actioned
  • improved: there is now a grayscale effect over the IDE window when the IDE loses the input focus to help identify IDE focus issues

BETA 425

  • fixed: C static library support was broken, causing a crash in BETA 424.  Demo SQLITE3 project available in discord

BETA 426

  • improved: rewrite of the semantic highlighting client side code for better scrolling performance [ fafalone, discord ]
  • improved: rewrite of the folding-range client side processing code for better performance [ fafalone, discord ]
  • added: command-ID tbEditor_CursorSelectLineStart assigned to shortcut key SHIFT+HOME [ Gary Miller, discord ]
  • added: command-ID tbEditor_CursorSelectLineEnd assigned to shortcut key SHIFT+END [ Gary Miller, discord ]
  • fixed: CTRL+S shortcut now triggers the tbProject_SaveAllChanges command as per previous versions, rather than tbEditor_Save [ FullValueRider, discord ]
  • fixed: File > Save Project command not working (tbProject_Save command replaced with tbProject_SaveAllChanges)
  • fixed: form Caption property was not being respected at runtime [ fafalone, discord ]
  • fixed: property sheet text box width not filling the available space [ fafalone, discord ]
  • fixed: form designer focus issue [ fafalone, discord ]

BETA 427

  • fixed: TOOLBAR panel cannot be reshown after closing it [ Kampfketer, discord ]
  • fixed: PROPERTIES and EDITOR panels would not save/restore correctly as floating panels [ Kampfketer, discord ]
  • fixed: (regression) code editor was not auto-closing parenthesis [ fafalone, discord ]
  • fixed: (regression) compiler crash on parsing some type libraries such as RC6 [ jpbro, discord ]
  • fixed: (regression since BETA 425) no semantic highlighting shown for references
  • fixed: Form.Controls.Add() third parameter is now Optional [ EduardoVB, discord ]
  • fixed: reversed power operator arguments for non-constant folded results [ https://github.com//issues/1741 ]

BETA 428

  • improved: a few minor IDE css tweaks [ sokinkeso, discord ]
  • fixed: (regression) 'PREVIEW' button on the toolbar not working
  • improved: some tweaks to the Light theme CSS to improve contrast and visibility of some IDE elements [ https://github.com//issues/1751#issuecomment-1912398419 ]

BETA 429

  • fixed: tbProjectExplorer_SelectedItemsDeletePermanently command not working (from menu/shortcut) [ jbpro, discord ]
  • fixed: tbPackagePublishing_HidePanel command not working
  • fixed: (regression since BETA 426) a code document without any foldable blocks could crash the compiler when opened in the IDE
  • added: context menu for the Debug Console that offers 'Copy All' and 'Copy All (With Timestamps)' options (commands tbDebugConsole_ClipboardCopyAll and tbDebugConsole_ClipboardCopyAllWithTimestamps) [ fafalone, discord ]

BETA 430

  • fixed: (regression since 424) potential crash when opening some projects
  • fixed: if the PROBLEMS panel has been closed, the IDE will fail to unload a project [ https://github.com//issues/1752 ]
  • improved: the auto-scroll lock on the debug console would sometimes cause the mouse wheel to be unresponsive initially until the lock was breached
  • fixed: (regression since 424) auto block completion was case-sensitive [ https://github.com//issues/1504 ]
  • improved: bad saved panel layout data now wont lead to having an unclosable docked panel [ fafalone, discord ]
  • fixed: (regression since 424) SHIFT+SPACE / SHIFT+BACKSPACE were not working in the code editor
  • added: Sample 17   Static Libary Example (SQLITE3) [ fafalone, discord ]