The BricsCAD Lisp Developer Support Package - Torsten Moses Reveals It All

 

Bangalore, India

Current promotion: GeoTools V14-99$, CADPower-39$. Read more....

Last week, Jason Bourhill of CADConcepts, New Zealand and I had a very long conversation with Torsten Moses, spanning 5+ hours over a TeamViewer and Skype session. This is going to be a very long post about what we learnt from him. Jason provided significant inputs to this blog and helped me make this even better than what was my first version. Thank you, Jason.

For those who do not know Torsten, he is the guy who created the Lisp engine in BricsCAD, and he is one responsible today for its upkeep and growth in BricsCAD.

Some background: Lisp has traditionally been the first and oldest API for all .dwg-based CAD systems. It is still a very popular programming platform for CAD with a lot of old-time programmers still actively using it. Autodesk has long been trying hard to 'kill' Autolisp and wean programmers off it. To achieve this, they stopped adding new functionality to Lisp since the last decade. However, the popularity of Lisp never decreased. Newer methods (using the ActiveX and ARX-based connections) and workaround techniques ensured that Lisp programmers never had to suffer from a disadvantage while developing apps. New, full-blown apps continued to be developed.

Last month, I did a benchmark to compare the Lisp speeds in different CAD systems and BricsCAD was on top of the chart. This means that Lisp in BricsCAD has been actively optimized and built-to-speed while in other CAD systems, it was left to rust on an antiquated Lisp engine that never kept pace with other changes happening around it. During the Bricsys 2012 Annual Conference in Amsterdam, it was announced that there would be a new suite of developer applications for Lisp programmers. The Lisp Developer Support Package (LDSP), to release during the life-cycle of BricsCAD V13 would provide professional software developement, debugging and peformance analysis tools for Lisp programmers.

This post is a curtain raiser before the actual announcement from Bricsys and provides a view of some of the new features you are going to see in the BricsCAD LDSP. Please note that this is still work in progress. What you are going to see in the release is expected to be much more than what I report today.

The best place to start is to look at the CHM documentation and you will know what to expect in LDSP (see below):

Fig 1: The Lisp Developer Support Package Main menu

Torsten began his story by going back a little in time and by telling us about some of the brainstorming sessions with the BricsCAD development team - how he was sure, clear and insistent about wanting 100% application level compatibility in BricsCAD compared to AutoCAD. He told us that the key differentiator for BricsCAD would be when, at its API level, it would be fully compatible, predictable,  stable and at par or exceed AutoCAD.  There is no better proof today that this has been achieved - BricsCAD has nearly 300 third-party applications which is the highest for any .dwg CAD platform other than AutoCAD.

Significant improvements to the BricsCAD Lisp engine have been happening since V12, but this was not documented well or announced publicly. The LDSP will document all of these and put them together under one umbrella. It also introduces new tools, enhancements and features never seen before.

Some of those major enhancements since V 12.2 are:

• more LISP core functions and functionality
• better overall performance of LISP execution
• improved compatibility with AutoCAD AutoLISP

"During V12 lifetime, all these new and additional features have been stabilised, and have reached a safe and reliable stage with V13 now, and is fully covered by both regression and performance tests." said Torsten.

Updated OpenLisp Core (since V12.2)

A new Lisp engine was introduced in BricsCAD V 12.2. The former AutoLISP -> OpenLisp PreProcessor was removed. This resulted in :

• faster Lisp loading by 30% to 40%
• no Lisp code growth any longer, which reduces Lisp and System memory load
• AutoLISP code execution improved by approx. 5-10%; less GarbageCollections

Support for ObjectDbx.AxDbDocument COM interface (since V12.1)

"The request for this came from LISP developers", said Torsten.

The ObjectDbx.AxDbDocument interface has been added to LISP engine:

• allows Lisp apps to open/read/write dwg files without an associated "editor document"
• compatible with AutoCAD-based AutoLISP code, using "ObjectDbx.AxDbDocument" COM interface

"Fast-COM" layer for Windows and Linux (since V12.2)

Torsten told an interesting story about Fast-COM. When Bricsys started developing BricsCAD for the Linux environment, there was a challenge on hand to provide all the ActiveX functions, like (vla-), (vlax-) and (vlr-). Linux does not have a COM interface into which you can pipe your requests and get results. As a result, Torsten had to write native equivalents for ALL of the ActiveX functions which would mimic exactly what the COM-based functions did. This was done by building a direct bridge into the CAD kernel.

On Windows, COM was the normal way to execute these functions. BricsCAD is built on an OS-neutral code and the new, direct COM-removed functions would run in Windows as well, just as it did in Linux. And Torsten found that it runs much faster than the COM based functions doing the same task.

Bricscad’s Fast COM intercepts all Visual Lisp calls, processing them within the BricsCAD environment, providing a massive performance improvement. Fast COM supports over 200 properties and 50 methods, and is available from all OS’s that BricsCAD Supports. In Windows, Visual Lisp calls not yet implemented as Fast-COM is passed on to Windows COM to be process in the traditional manner.

Thus, Fast-COM was born, and is now active in BricsCAD by default. So, the next time you run something like (vla-intersectWith oname1 oname2 extFlag)in BricsCAD, you may think you are using COM like in other CAD systems but internally a native Lisp implementation is doing the job and faster too.

"Currently, there are > 200 Properties and > 50 Methods implemented in Fast-COM, and more to be done in due course of time", said Torsten Moses.

Here are some key bullet points about Fast-COM:

• bypasses Windows' COM execution layer (performance)
• bypasses most data marshalling as required by Windows COM (memory load)
• fully enables identical COM-based (vla-) client LISP code under Windows & Linux
• significant performance improvement for (vla-) function calls (up to factor 30 and more), at same level as ARX/BRX

The last point is extremely note-worthy. same level as ARX/BRX - this is going to be music for Lisp programmers. It is not difficult to guess why. Both COM and .NET are wrappers, they percolate data through layers of indirection to get to the root. FAST-Com, on the other hand, is like a native BRX application running Lisp. The benefits of speed are obvious (see Lisp Benchmark blog post).

VLE LISP Function Library (since V12.2)

Recently, BricsCAD extended LISP further through the In addition of the VLE function library.

The VLE LISP Function library works on many levels:

• Extends LISP through additional functions. VLE LISP has functions to assist with working with Lists, Entities, Strings, Files, Maths, Datatypes, and more. This offers a significant saving of time to users new to LISP, as they don’t need to create these functions for themselves. For existing LISP users it offers a way to improve the performance of existing code.
• Optimises existing LISP by using pattern matching techniques, automatically swapping out inefficient LISP code for optimised internal code. This happens transparently at run time. There is no need to make any changes to existing code to take advantage of it.
• Enhances speed when performing transactions with the drawing database, such as the creation and manipulation of objects on the drawing, and within definition tables.
• Is cross CAD application compatible. Provides emulation of VLE LISP functions through vle-extension.lsp, allowing LISP developed using VLE calls to work on other CAD applications that provide support for LISP.
• Is cross OS compatible. VLE is available in Linux, Windows, and Mac (when it arrives).

In addition, the VLE LISP Function library provides an alternative set of functions for every inefficient Lisp or Visual Lisp function(s). For example, entget and append are replaced with a vle- prefixed function of the same name which is must faster. As a Lisp programmer, I have been using entget and append for a long time and changing the code to a (vle-entget..) and  (vle-append..) will make my code faster and efficient.

You can view some of the efficient alternative functions here:

Fig 2: The VLE Lisp library Main menu

 

Fig 3: The List processing functions

Fig 4: The VLE String, File, Math and DataType functions

Fig 5: The VLE Utility and Transaction functions

This is what Torsten summarized about the VLE LISP library:

• currently provides > 35 new and efficient LISP core functions, subject to further extensions (feedback and suggestions are welcome)
• all functions are also provided for other CAD systems (and older BricsCAD!) by emulation "vle-extension.lsp"
• by its built-in nature, VLE functions provide both performance improvements (20%  to 20x) and reduced memory load.

I liked the third point - provided for other CAD systems - by emulation "vle-extension.lsp" - you can write your Lisp code which will run using the VLE library in BricsCAD and will run under other CAD systems as well. You have truly portable code, even if it is not as fast in other systems.

LISP Optimiser (since V12.2)

The new built-in LISP Optimiser recognises inefficient, slow LISP code patterns at load-time and replaces those patterns with more efficient code. This helps overcome several bottlenecks caused by missing AutoLisp features.

These are the key take-aways from Torsten:

• works completely transparent without any specific requirements or interaction
• increases LISP performance (for affected code) by 20 - 2000 %
• reduces LISP memory load and amount of GarbageCollections
• number of recognised code patterns is subject to further extension

LISP Profiler (since V12.2)

A Lisp profiler has been added in BricsCAD on request from Lisp developers. The Lisp Profiler provides tools for troubleshooting, performance and code analysis. The Profiler can be controlled manually, and also at execution time. Results are provided on commandline and as a log file.

It is currently functional, providing a lot of information about the entire code execution. New features like a GUI interface and the ability to save and load profiled data sets will be added soon.

Lisp Benchmarks: I wrote an article about the Lisp execution speed in BricsCAD. The entire benchmark suite is now available in BricsCAD as part of the developer package.

Initially developed for private use, the Lisp Benchmark suite grew over time to become an assisting tool even for developing the BricsCAD LISP engine. The Lisp Benchmark is still under development, and all major work, new functionality and improved Lisp engine implementations are covered by the benchmark system.

The LISP Benchmarking Utility is a standlone FreeWare utility, as long as all copyright notes remain unchanged throughout all its files.

Benchmark Concepts and Features

The Lisp Benchmark is designed to run on any LISP enabled CAD system, providing an AutoLISP compatible language and environment on Windows, Linux and Macintosh platforms. It contains more than 360 individual benchmarks, grouped in 8 categories, covering the following areas:

• Standard AutoLISP functions: as defined by the AutoCAD R12 standard
• VisualLISP functions: (vl-xxx) functions provided by VisualLISP since AutoCAD 2000
• COM utility functions: (vlax-xxx) functions
• COM interface functions: (vla-xxx),(vla-get-xxx) and (vla-put-xxx) functions
• VisualLISP Extension functions:  verifying all of BricsCAD's (vle-xxx) functions, and emulated on other systems
• Fast-COM utility + interface functions:  verifying BricsCAD's "Fast-COM" mode - emulated on other systems
• LISP-Optimiser verification benchmarks verifying BricsCAD's "Lisp-Optimiser", emulated on other systems
• User-defined (complex) benchmarks  complex scenarios based on user feedback & demand

Each individual benchmark file examines a particular, single LISP function in typical usage (different arguments etc.); this verifies the low-level, implementation-specific performance of the target function.

For all categories a specific command is provided to run all the benchmarks of that category; these commands also create a CAD-System + Platform + Command specific log file to store benchmarking results.

Besides predefined benchmark commands, each individual benchmark Lisp file defines a C:Run command to run separately - the easiest way to benchmark the particular function.

DESCoder Lisp Encryption

The DESCoder is an encryption, packaging, compression and code-securing tool for Lisp applications. You can use the DESCoder to create a packaged project similar to a Visual Lisp project, and can combine LSP, DCL and other support files together into a package with the extension .DES.

DEScoder provides strong BlowFish encryption for AutoLISP files. They can be used in BricsCAD only.

"DESCoder is for beginners wanting to secure just one single LSP or for use in a professional software development organization", said Torsten, and he told us about some of the advanced features (see below).

• Drag & drop support for all relevant dialog controls
• Command line support for batch processing
• Wildcard support for file specification in command line mode
• Program dialog is resizeable and persistent (size and positions are maintained across sessions)
• Single file history for last-used 30 Lisp source files for quick access to most often used Lisp files
• Project history for last-used 30 projects for quick access to most often used projects
• Automatically reload last-used project at startup, to quickly rebuild with a single click
• Tool tips provide assistance for beginners, can be disabled

Support for UltraEdit Editor (A credible alternative for Visual Lisp editor and development environment)

At the time of this writing, support for UltraEdit Editor was not yet complete but very much in design and development. Ultra Editor is a very powerful ASCII, hex and HTML editor with text color coding, bracket matching,scripting, macro-creation and third-party plugin development capabilities. All of this makes it an ideal tool to customize and extend it as an editor for Lisp programs. For example, it would be possible to call the DEScoder to generate encrypted Lisp programs from within the Ultra Edit environment. Those of you who have been wanting a Visual Lisp editor-like tool in BricsCAD now have something cool to look forward to.

With many good programming based editors available (some of them even free), Torsten saw no value in re-inventing an editor specifically for the BricsCAD environment. Most already had formatting and language tools for LISP. The main thing missing was the ability to perform debugging tasks. Debugging LISP in BricsCAD is already generally easier, as it returns the more information on errors than in other CAD systems, such as location of the errors in the stack. Because of the power of Ultra Edit, Torsten sees the potential to provide a LISP debugging tool for it in the future.

Discover BricsCAD, discover GeoTools-AutoCAD , GeoTools-BricsCAD, CADPower-AutoCAD, CADPower-BricsCAD, discover TechCenter. Brand new ways to work smarter in CAD!!

Comments

[Kotaa Krishna Kamath]

Very nice Article must read for all Lisp developers

[ralphg]

Thank you for the extensive update. It is nice to see a CAD vendor doing what its customers want, rather than trying to direct them towards untested or unwanted futures.

[Bill DeShawn]

Why am I have an age-old LISP problem with (command "_.line" pause pause "")? Well, I got out of BricsCAD and got back into it, and now it's just fine. Now I know I might have to do that from time to time.

[Rakesh Rao]

Bill,

That should not happen from time to time. If it has happened any time, it means the Lisp engine probably got unstable because of some other part of your code which you need to check. The line of code that you show above looks perfectly fine.

Best Regards
Rakesh Rao