Linx 4.0.928 released

Release notes

• Add socket timeout to FTP client
• Fix PDFConcat changes readonly fields to editable
• Fix add content length to REST webservice call using POST
• Fix FTP access denied error when changing to root directory on login
• Add PDFProtect component
• Fix corruption of runtime info on LinxServer by changing to json format
• Update to latest Validation tool
• Make stored procedure name editable
• Fix LinxServer multiple upload file problem
• Fix Oracle stored procedure execution
• Remove "Integrated security" from Oracle connection string editor
• Add additional proxy settings on web services components
• Fix bug: If a Linx service (e.g. DirectoryWatch) can't start, it kills the host process
• Fix bug: Object reference error when trying to restart faulty DirectoryWatch service

Linx 5.3.136.7843 released

In this release we've improved our logging, added more tracing info to the debugger, made updates possible from the Linx Server UI, added licensing infrastructure and fixed numerous bugs. 

Seasoned Linx users will find the logging feature interesting. When running the debugger you get the following output by default:

By right-clicking on a function you can enable logging to give you this:

So now you can get detailed information about what the function did during execution. Just remember that enabling this on your query that returns a googolplexian results might make you miss dinner.

Release notes

• Fix null reference exception.
• Add ToInt64 extension methods on decimal and double.
• Debugger shows incorrect values after recursive call.
• Update Linx plugin.
• Call UpdateToLastestVersion method when loading function and service data.
• Add new updating mechanism to functions.
• Add update functionality for PassAsReference properties.
• Fixed reference not found error.
• Remove binding errors on ProcessControl.
• Do not override setting values from debugger.
• Use distinct assemblies to speed up compilation.
• Scroll current debug target into view.
• Plugin manager should tell us when server not contactable.
• Updates from Linx Server UI.
• JsonIgnore non-persisted properties in SolutionRuntimeInfo.
• Fix JSON/XML expression formatting in collection editor.
• Add custom type validation on property value.
• Fix version binding.
• Allow multiline Input debug values.
• Fix DummyItem displayed when debugging.
• Fix type reference change in SetValue.
• Improve logging infrastructure.
• Output log.
• Debug Output: Add Trace information.
• Fix app.config.
• Fixed deadlock when Stopping debugger.
• Fix Paste after copy not taking selected function into account.
• Linx Server RunProcess window: Fix AppState is not defined error.
• Fix Property dropdown: Dropdown was empty for property of type IEnumerable<byte>.
• Debugger: Dispose faster.
• LinxServer: multi-line parameter input and output when running a process.
• Return empty server package list if internet connection fails.
• Store: Better error message when function data update fails during solution load.
• Fix BindingExpression error "System.Windows.Data Error: 8 : Cannot save value from target back to source.".
• Add ToBytesFromBase64 as extension method on String.
• Tell the user when loading a Linx 4 file.
• Add ToBase64 as extension method on List<byte>.
• Fix BindingExpression errors "Cannot find source for binding with reference 'ElementName=expressionEditor'" and "BindingExpression path error: 'Children' property not found on 'object'".
• Fix property reference bug.
• Fix "ValueIsValid" binding error.
• Fix SetValue bug - loses value when target is removed.
• Change display of CustomTypes in property window.
• Fix list of custom type breaks when custom type is renamed.
• Change display of CustomTypes in dropdowns.
• Allow the use of CustomTypes across projects.
• Debugger: Do not display items that do not have debug values.
• Debug Values: Show dates in local culture.
• Double-click on Linx 4 file will launch Linx 4.
• Designer: Add licence page.
• Fixed EmptyFunction discarded property value-type info.
• Designer: Loading a Solution without the required Plugins installed should tell the user.
• Don't allow saving the solution when plugins are missing.
• Change upgrade detection in installer.
• Fix immutable functiondata and servicedata does not add version.
• Log full exception text in logger.
• Lazy load recently used functions.

Linx 4 to 5 converter

Today marks the first release of the Linx 4 to 5 converter.  This application will take an existing Linx 4 solution (and optional config) file and create a corresponding Linx 5 solution file from it.  As part of the conversion process, the user will be presented with a conversion log that can be used to track down any conversion issues.

How do I use it?

The converter is a command-line application (Convert.exe) that accepts several parameters.  They are:

• L4Solution:  Path to the Linx 4 solution file that you want to convert.
• L4Config (optional): Path to the Linx 4 configuration file that accompanies the solution file.
• L5Solution: Path to the Linx 5 solution file that will be created.
• Log (optional): Path to the conversion log file that will be created.

For example:

Convert.exe L4Solution="C:\Temp\OldLinx4Solution.lsoz" L4Config="C:\Temp\OldLinx4Solution.lsoz.config" L5Solution="C:\Temp\NewLinx5Solution.lsoz" Log="C:\Temp\Conversion.txt"

While the application is running, it will display the current conversion status on the screen.  If you've specified a log file (using the command-line parameters), this information will also be captured to disk.  Conversion statuses are as follows:

• Verbose: Display information about the program flow, e.g. the component being converted, loading of the solution files, instantiating the Linx 4 and 5 environment, etc.  Verbose messages are displayed in the default command-line colour.
  
• Information: Display information about non-breaking changes that were made to the converted solution.  Linx 5 for example follows a more strict naming convention than Linx 4 and invalid names have to be converted to ensure a successful conversion.  These messages can also include information about properties that were ignored or replaced with a new Linx 5 equivalent.  Information messages are displayed in green.
  
• Warning: Warnings messages are displayed when a component can only be converted up to a point, but requires some user intervention, either on the Linx 4 or Linx 5 side.  Warning messages are displayed in yellow and are repeated at the end of the conversion for convenience.
  
• Error: Used to display a fatal conversion message.  If an error is encountered, the whole conversion will stop.  Ensure that you have no validation errors in your Linx 4 solution before you start the conversion.  Error messages are displayed in red.
  

The small print

The converter will (for now) only convert the most commonly-used components.  Components that do not contain a Linx 5 equivalent will be included as a missing function in the Linx 5 solution - you won't however be able to use these solutions until you remove the applicable component.  Also, be sure to check for any validation errors/warnings in the converted Linx 5 solution after the conversion.

The converter does a one-to-one conversion of the Linx 4 solution and does not try to understand the solution logic.  You may therefore end up with a lot of extra components that can sometimes be consolidated into one Linx 5 component, e.g. separate assignments can be done in a single expression. 

We want to hear from you

As mentioned, this is the first version of the converter.  There are a lot of Linx 4 solutions out there that does a lot of different things, so please let us know if you have any questions, suggestions, feature requests, missing conversions. etc.

Linx 4 vs Linx 5: Part 6 - Plugins

Linx 4 is distributed with a single installer that contains the Designer, Server and all Components. With Linx 5 the Designer and Server is distributed with a single installer and adds the concept of a Plugin that is distributed separately. Linx 5 introduces the following changes

• Components are now called Functions.
• Plugins are managed with a Plugin Manager.
• Plugins contain Functions and Services.

While this mechanism is much more complex than the single installer model it has the following advantages

• Plugins can be added, updated or removed without affecting the core Linx 5 applications.
• Third parties can contribute Plugins without affecting the Linx 5 codebase.
• Users can selectively install functionality.
• We can have different licencing models for Linx 5 and Plugins.

This is what it looks like

and here is the Plugin Manager

Linx checks on startup for new versions of Plugins and shows a notification if any are found. Clicking on Install or Update in the Plugin Manager will download and install the selected Plugin.

You can see the list of available Plugins here. Please submit any questions or suggestions to our Linx Community at http://community.linx.twenty57.com/.

Linx 4 vs Linx 5: Part 5 - Deploy

Linx 5 allows the deployment of a solution to one or more Linx Servers straight from the Linx Designer.

With the solution you want to deploy open, click the Deploy button on the toolbar.

Select the Linx Server(s) in the Deploy window and click Deploy. A notification top right will show that deployment is happening.

When deployment is finished the same notification section will change to show the finished state.

Navigate from your Linx Designer to the Server UI by clicking on the link in the notification window.

Add, edit and remove servers from your deployment list from within the Deploy window.

Please submit any questions or suggestions to our Linx Community at http://community.linx.twenty57.com/.

Linx 5.3.51.7491 released

This release has mostly been about bug fixes and the new Linx web site. We've also added the ability to use Settings in Linx Service Properties. This allows you to change a TimerService property without having to upload a new Linx Solution. Other Linx Services have also been updated to use this new capability.

Changelog

• Fix bug: LinxServer: Process output failed to show.
• Fix bug: Embedded custom type compiler error.
• Show Plugin Manager with Updates selected.
• Add ReferencesChanged method for applicable components.
• Limit objects available in property dropdown to cater for functions like BeginTransaction and the functions that hooks onto its output.
• Update plugin.
• LinxServer: Do not recompile when Settings change.
• Service properties can reference Settings.
• Fix bug: Type-literal compile error.
• Fix bug: Match service by Id rather than Name.
• LinxServer: SolutionHost recompiles after plugin updates.
• Fix bug: Check for null WPF listview item.
• Fix bug: Move IsExpanded property to base to prevent binding errors on treeview.
• LinxServer: Delete next version folder if it fails to create. Do not fail on non-existent settings file.
• LinxServer: Handle null activeServices array.
• Fix bug: DummyDomainItem shown in properties when nothing selected.
• Only add resolve paths for valid components.
• Fix bug: Object reference error on Expression Editor intellisense for invalid properties.
• Fix colour in Expression Editor.
• Fix bug: Recursive RunProcess crashes when debugging.
• Fix bug: Octal conversion error in debug statistics.
• Fix bug: Change Guid.ToString format to fix object reference error.
• Fix bug: Double slashes in help links.
• Show validation error when CustomType deleted.
• Update help and release notes links to point to new site.

Linx 4.0.905.4143 released

Changelog

• Fix bug: Similar process names caused component context to clear unexpectedly
• Fix bug: Recursive RunProcess call does not map DataIn values correctly

As you may have noticed, this is the second release in a week.  One of the fixes that we've made in the previous release ("DbRead keeps datareader open when code crashes in execution path") highlighted a caching problem that has been around for more than 2 years.

Linx uses an internal cache on some components to speed up execution, e.g. keep Excel open while writing in a loop.  When calling another process using the RunProcess component, the cache got cleared unexpectedly if the name of the process that you are calling is a substring of the name of the process that contains the RunProcess component.  E.g. process "ClearFiles_Outer" calls process "ClearFiles" using the RunProcess component.  Thanks goes to Luka Milutinovic for finding this very obscure error.

The second bug fix relates to the DataIn mappings of a recursive RunProcess.  If a process calls itself (using the RunProcess component), the DataIn values were not assigned while calling the process recursively.  Thanks goes to Carmen Lawrence for discovering this.

If you have already installed the previous release (4.0.902.4134), it is advised that you upgrade to this version.  If you have any additional questions, please do not hesitate to contact us.

Linx 4 vs Linx 5: Part 4 - Types

Type

A Type is a classification identifying one of various types of data such as String, Integer or Boolean. In Linx 4 we have a Variable component that can create one or more variables of a specific type. In Linx 5 we drop a Type on a Process to create a variable of that type.

Custom Type

Users can define their own Custom Type with the properties of their choice. Custom Types appear in the Solution Explorer.

To create a variable of a Custom Type drop it on a Process. The value of a Custom Type variable is defined in Json. There is an editor to make this easy.

List Type

A variable of type List has both a name and a content type. The content type can be any valid Type including Custom Types.

Values of List variables are defined using Json. There is an editor to make this easy.

Other Types

The other types are self explanatory - Boolean, Byte, DateTime, Decimal, Double, Integer (64bit) and String.

Linx 4 vs Linx 5: Part 3 - Expressions

An Expression is a combination of values, operators and functions that are evaluated to produce a single value. Expressions in Linx use C# syntax. Here are some examples

• 5 evaluates to 5
• 5 + 2 evaluates to 7
• "abc" evaluates to "abc"
• "abc" + "def" evaluates to "abcdef"
• 5 == 5 evaluates to true
• 5 != 5 evaluates to false

Any Property in Linx that does not need its value resolved at design-time can have an Expression as its value. Expressions are resolved at run-time. Expressions are denoted by "=" before the Expression.

Expression Editor

The Expression Editor makes it easier to create and edit complex expressions. It includes syntax highlighting, intellisense, code completion and built-in help.

This is what it looks like

Variables

a Variable is a value that is available at run-time. Variables are produced by

• Functions e.g. TextFileRead.
• Types e.g. String.
• CustomType e.g. Person with properties Name, Surname, Age.
• System e.g. CurrentDateTime, NewLine (CrLf) and Null.

Expression Editor System Functions

System Functions can be performed on specific types of data. They are typically used to do formatting, conversion or extraction. Examples are

• String - Contains, ExtractString, FormatWith, Trim.
• DateTime - AddDays, ExtractDay, ToString.

String Functions

DateTime Functions

Next

The next post will discuss Types, also brand new to Linx 5.

Linx 4.0.902.4134 released

Changelog

• Add compression to PDFExtractImages
• Add range and cell validation on ExcelRead
• Update validation dll
• Display invalid user error when calling Linx service with invalid credentials
• Installer now does not change Service Account in upgrade mode
• Improve the validation experience for massive solutions
• Change ExceptionHandler validation warning to error
• Allow case-insensitive upload of solution files
• Add SFTPList component
• Add SFTPUpload component
• Add SFTPDownload component
• Db components now supports stored procedures from Oracle
• DatabaseRead now does not clear output when SqlEditor fails to run query
• Fix bug: FileWrite with append and CloseAfterEveryWrite set to false gives closed error when writing to multiple files
• Fix bug: DbRead keeps datareader open when code crashes in execution path

Linx 4 vs Linx 5: Part 2 - Linx Designer

Here are some of the UI changes we've made.

New look, new features, some changes

Solution Explorer

Processes, Services, CustomTypes and Folders can all live under a Project or Folder.

Process Input and Output

Input and Output are now available from the Process Properties.

When a Process is dragged onto a calling Process

the Process Input fields are available as Properties.

Settings

Constants = Settings. Setting values can reference other settings.

Undo

Next

So far everything should be fairly familiar to Linx 4 users. Although the UI looks different the principal elements remain the same. In the next post we'll discuss something brand new - Expressions.

Linx 4 vs Linx 5: Part 1 - Overview

This is the first post in a series that explains the differences between Linx 4 and Linx 5.

Overview

With Linx 5 we've aimed to improve the user experience, technical framework and extensibility of Linx in order to give our existing clients a better product and to make it possible for us to reach a wider audience. Here are some of the visible changes.

Core functionality

• UI - The Linx 4 Developer is a 15 year old design. Linx 5 Designer gives it a modern look and allows us to extend it in ways that was not feasible with Linx 4.
• Undo - Undo and Redo.
• Expressions - Properties can be filled with Expressions like Price * 0.14 or ReadFile.Content + " THE END".
• Contextual Properties - We only show the Properties that matter. Changing a Property may result in other Properties being shown or hidden.
• Types - Types and Custom Types replace the Variable component. They allow for re-use and more flexible Process designs.
• Scoped Functions - Functions that require shared resources are now possible. Examples include DbBulkCopy and FileOpen.
• WebService Services - Design your own SOAP or REST web services without linking it directly to a Linx Process.
• Deploy - Deploy to multiple servers from within the Linx Designer.
• Plugins - Linx components are now distributed and installed as separate Plugin packages.
• Auto update - The Linx Designer notifies you if there are updates available and allows you to install it.
• Help - Online help available from within the Linx Designer.
• Community - Help each other use and improve Linx - http://community.linx.twenty57.com.

Who moved my cheese

Some components have been replaced by others or its functionality has moved to other areas in Linx.

• Components are now split into Functions, Types and Services. They are distributed via Plugin packages.
• Constants is replaced by Settings and are now at the Solution level.
• Variable is replaced by Types and CustomTypes.
• DataIn and DataOut are replaced by Input and Output which are properties on a Process.
• Assign is replaced by SetValue and Expressions.
• RunProcess is gone. Drag the Process from the Solution tree to the Process tab.
• FileWrite does not have a 'Close after every write' property. Use the FileOpen in conjunction with FileWrite or SetValue to keep a file open while writing to it.
• Use StringBuilder in conjunction with SetValue to build large strings. Concatenating to the same string in a loop will be slower than using StringBuilder.

New components, so far

• SOAPWebService and RESTWebService - Design your own or mimic another web service.
• DbBulkCopy - Quickly populate a database.
• Zip and Unzip.
• Finswitch - Interact with the Finswitch web services.
• MongoDb - Read and write to MongoDb.
• Assert - Make sure what you get is what you expect.
• RazorTemplateTransform - Transform Razor templates to produce html, xml or text.
• GraphicsMagick - Do amazing things with images.
• AmazonEC2 - Use Amazon's EC2 service.
• GenerateHash - MD5, SHAxxx.
• FileOpen - Keep a file open while writing to it.
• DirectoryOperations - Copy, move, create, delete, exists.
• ThrowException - Create your own exception.
• PDFOperations - Fill form, split, concatenate, sign, add watermark.
• StringBuilder - Build a large string at speed.
• XmlPeek and XmlPoke - Extract or replace Xml snippets using XPath.

Next

The next posts in this series will describe each of these changes in more detail.

Warning: Linx 4 XmlParser might bite

Summary

The XmlParser from version 4.0.812 is strict when enforcing the schema provided. The change was introduced to speed up parsing performance but as a consequence fixed a bug that allowed the parsing of xml documents that do not conform to the schema.

How does it affect me?

XmlParser components with incorrect schemas will stop working after upgrading to Linx 4.0.812 or later.

What should I do?

If you upgrade Linx solutions from versions prior to 4.0.812 and you use the XmlParser component, make sure your schemas are correct. Test the XmlParser with a sample xml document. If it fails to parse fix the schema. You can also build the schema from the sample xml by using the Schema Editor and following these steps:

1. Menu: File - Build from xml...
2. Select the sample xml file
3. Menu: Process - Compile Schema
4. Accept
5. Select the root element

Thanks to Dina for spotting this.

Various Linx 5 plugins released

All Linx plugins have been updated to be compatible with Linx 5.3. The following plugins have additional changes.

Database 1.0.43.253

• ExecuteStoredProcedure: Detect connection type when changing connection string.
• UI changes
• ExecuteStoredProcedure: Fixed crash when setting connection string to a variable.

Excel 1.0.14.74

• ExcelRead: Fixed Range editor throwing exception
• UI changes

File 1.1.34.128

• UI Changes
• Add FileOpen function. This function allows you to keep a file open and write to it with one or more functions. It improves performance when dealing with lots of writes to a single file.

FTP 1.1.18.67

• FTPList: Add recursive property
• FTPUpload: Add CreateDirectory property

GraphicsMagick 1.0.23.60

• Update property grid external assembly
• UI changes
• Logical ordering of transform operation properties.
• Fixed MotionBlur and CDL Transforms

Text  1.0.24.59

• Update RegularExpression Editor
• UI Changes

WebService 1.0.44.183

• UI Changes
• Updated DynamicSoap.dll (Support non-bodywrapped message contract methods)
• CallSOAPWebService method editor: Automatically add missing '?wsdl' for .asmx and .svc 
• CallSOAPWebService: Fixed error when selecting method

Linx 5.3.1.7318 released

This release includes bug fixes and tweaks to the UI. The service start-up behavior has also been improved. If a Linx Service fails to start it will periodically try again e.g. a service that depends on network connectivity might fail to start if the network is down. By retrying periodically it will start once the network is up.

The following release notes only relates to Linx Designer and Linx Server and do not include new or changed plugins. In Linx 5 the plugins are updated separately from Linx. We will release separate notifications for new plugin versions.

Release notes

• UI changes
• Update Linx plugin
• Force a restart after doing package updates in the designer
• Change designer and server to be culture invariant
• Display function names in compilation errors
• Change service start-up behaviour to better handle solutions with broken services
• Fix bug: Double-click on lsoz file does not open file in designer
• Fix bug: Compiling process that is not open in designer gives object reference error
• Fix bug: Property grid does not display items in alphabetical order when that option is selected
• Fix bug: Character expression '"' was interpreted as the start of a string in expression editor
• Fix bug: Instance of custom type editor throws exception when not all properties are filled in
• Fix bug: Collapsed docked windows do not open on hover
• Fix bug: Server statistics incorrectly calculated when multiple threads are running
• Fix bug: Custom type property not parsed correctly when used in expression
• Fix bug: Validation index error for custom validations
• Fix bug: Copy paste of child items with references gives error
• Fix bug: Project paste not working
• Fix bug: ForEach output does not change when List type changes
• Various other enhancements and bug fixes

Linx 4.0.882.4078 released

Changelog

• Validator - Logging of record and line counts 
• Stop orphan SolutionHost processes on service stop and reload all solutions
• Allow user to set WCFReceiveTimeoutSeconds for long running processes (default 10 minutes)
• Fix bug: SplitString with repeat true and no configured fields go into endless loop

Linx 4.0.877.4069 released

Changelog

• FTPFileRename - Add CreateDirectory property
• FTPFileUpload - Add CreateDirectory property
• Validator - Add IgnoreLinesPattern property
• Change saving of solutions to be unaffected by regional settings
• Create log entry if SingleInstance process in service times out on the server
• Fix bug: CPU utilization statistics for host processes reported incorrectly
• Fix bug: Solution statistics calculated incorrectly under heavy load
• Fix bug: Extract string component with start position and end length can go into endless loop
• Fix bug: Cache compiled web proxy to prevent rising memory usage on CallWebService component
• Fix bug: Object reference error when creating documentation with null object value
• Fix bug: Layout file in use error when multiple solutions are started on server

Needle In The Haystack Award

The Needle In The Haystack Award goes to Byron for correctly calling out the CallWebService component as a memory thief and supplying enough information to reproduce the problem. Well done Sir.

Linx 5.0.680.6896 released

Today marks the release of another Linx 5 Beta (5.0.680.6896).  If you have an existing Linx 5 installation, your designer will prompt you to upgrade to the new version.  If you don't have Linx 5 installed yet, you can download the latest version from the Linx website.

It has been a while since our previous public beta.  A lot of work has gone into making the back-end more robust and we hope that you will notice the difference.  We will continue to make improvements and add new features using the feedback that we receive.

On the component side, we have improved on existing functions and services, as well as create a lot of new items (e.g. Zip, ExecuteStoredProcedure, etc.).  Head on over to the Linx help site to read more about all of these new (and existing) items, check out our video tutorials, or install the necessary packages in Linx and try them out for yourself.

In upcoming releases, you will (among other things) start to see some of the styling work that we are busy with, so please be sure to update when there are new versions available.

Linx 4.0.867.4040 released

Changelog

• Add ActiveDirectory component.
• Add PDFExtractImages component. This component is not distributed with Linx.
• Validator updated with performance improvements and stricter DateTime validation.
• Linx Server logs shown in grid style.
• Linx Server logs filtered by from and to dates.
• Linx Server header colour and name can be changed to make server identification easier.

Server name and header colour

It is now possible to specify a header colour and server name per Linx server instance.  This can be useful in environments where multiple Linx servers are hosted.  Please note that this functionality is only supported on modern browsers.

Linx 4 memory usage

Every now and again we get questions like "How much RAM do I need for my Linx Server?" or related questions about how Linx uses RAM. This post tries to shed some light on these questions.

Infrastructure

Linx uses the automatic memory management supplied by the .Net framework. Memory is allocated to Linx as required and freed by .Net using a garbage collection algorithm. The garbage collector balances the memory requirements of the operating system with the cost of freeing and allocating memory. You can read more about the technical details here.

Linx Server

The Linx Server hosts each Linx Solution in its own Windows process. Each solution takes up approximately 34MB of RAM for its basic operations. RAM usage above that depends entirely on how the solution is implemented. One can implement the parsing of a big text file and only use 2MB of RAM or you can implement the same thing slightly differently and use GBs of RAM.

Components that may use a lot of RAM

FileRead

Reading files where OutputToTable = True or ReadType = Complete uses much more RAM than reading files line by line and not outputting to a table. The amount of RAM consumed by using ReadType = Complete or OutputToTable = True is directly proportional to the size of the file that is being read.

Database read components

All database components  where OutputToTable = True or GetAllDataToClientFirst = True use much more RAM than when both are False. Either one of these properties, when set to True, will read all the data from the database and store it in memory.

Components where OutputToTable = True

All components where this property is True will collect all the data in memory.

Assign

Assigning to a string variable where the operator = Add. This keeps on increasing the string size in memory.

NewRow

Adding a new row to a table variable increases the memory used by the table.

Image processing components

Image processing components may be RAM and CPU intensive depending on the type of function that is being performed. Test these in separate solutions to determine the effect your particular implementation has on RAM and CPU.

Multiple processes running at the same time

Linx can run multiple processes at the same time. None of the processes might use a lot of RAM on its own but together they can have a significant effect on RAM usage e.g. A directory watch service fires an event for a new file. Each file takes 5 seconds to process and uses 10MB of RAM. Imagine dumping 100 files in the directory in 4 seconds. That's 1GB of memory required for at least a second.

Testing some scenarios

I tested some of the scenarios mentioned above. The tests all use the same text file of 10MB with 10000 lines, each 1000 characters in length.

• BuildFileInMemory builds a string using a string Variable and the Assign component and then writes it to a file.
• BuildFileOnDisk builds the same file but writes it to disk line by line.
• ReadFileComplete reads a file with ReadType = Complete.
• ReadFileLineByLine reads the file line by line.
• ReadFileLineByLineToTable reads the file line by line but OutputToTable = True.
• StoreDataInTable reads the file line by line and then uses NewRow to add it to a table Variable.

The results

1. Building a 10MB file in memory uses 99MB RAM vs 3MB for the direct to disk version.
2. Reading a 10MB file completely uses 42MB of RAM vs 10MB if read line by line.
3. Memory was not immediately freed up by the garbage collector on my system but simply re-used when I ran the same thing again (if my system required the RAM it would have been freed).
4. The garbage collector frees up memory as the rest of my system requires it.

A common reason to use RAM rather than go directly to disk is speed. Here are the results of two of the tests from above writing to a SSD and a USB drive.

Using memory to build up a large string and then writing it to disk as one item is approximately 60% faster than going repeatedly to disk on a very slow USB drive. But it still only takes 1.5s to write a 10Mb file to a lowly USB drive. My requirements will have to be pretty extraordinary to justify building large strings in RAM for speed reasons alone.

Linx Designer

Do not test RAM usage by using the Linx Designer. The debugger stores a lot of additional data for debugging purposes that will give you false results. Test the RAM and CPU usage of a solution in a Linx Server.

Things to consider when writing Linx Processes

• Number of files processed.
• Size of the files.
• Concurrency. Can processing happen simultaneously?
• Number of database transactions we are reading.

If any of these are large numbers try to use RAM sparingly by taking into account the comments above. If you are unsure experiment by putting the processes you are concerned about in its own solution and test it on a Linx Server.

How much RAM do I need for my Linx Server?

Follow one of these strategies to determine an answer:

1. Look at similar implementations and see what works for them.
2. Write a solution to simulate the load and try it on a Linx Server.
3. Take as much RAM as possible. RAM is cheap, your time is not.

Linx 4.0.858.3983 released

Changelog

• ReadMailMapi now returns empty string instead of null for emails with no body
• LinxServer Service icons changed
• LinxDesigner scaling of fonts for different screen sizes improved
• CallSoapWebService now supports a union structure for a message request
• CallSoapWebService improved handling of dates and times
• Validator add AllowExtraFields property
• Validator add LogEveryLine property
• DecodeFormzBarcode can now distinguish between Signature and other barcodes
• Add Ctrl-A shortcut to Text Editor
• LinxServer fallback to file upload dialog for IE11 when using NTLM authentication.
• Fix bug: CallSoapWebService not showing list/array in correct format
• Fix bug: CallSoapWebService incorrect or missing use of XmlElementAttributes
• Fix bug: CallSoapWebService empty xml request string
• Fix bug: Validation of disabled Service Events
• Fix bug: Validator not correctly validating a DateTime
• Fix bug: Validator using previous row values
• Fix bug: DateTimeFormat using incorrect CultureInfo
• Fix bug: Validator shows error when no delimiter specified for fixed length format
• Fix bug: RabbitMQService properties could not be set to Constants
• Fix bug: RabbitMQService crashes when the Rabbit Server is down

Validator

We've made some major changes to the Validator and there are still more on the way. Please let me know if you are using it. I would like to include your usage scenario in our tests so we don't inadvertently break something...