.

Release Notes: CODESYS Test Manager 5.2.0.0

Key Issue Type Summary Resolution Note
TM-1938 Improvement

Support File Based Storage in LoadProject Test Actions

Fixed [[GENERAL]]
The test action 'TestManager.Projects.LoadProject' now also supports CODESYS File-Based Storage projects.

Minimum requirements:
* CODESYS V3.5 SP22
* CODESYS File-Based Storage 1.0.0.0

Test script editor:
If CODESYS File-Based Storage is not installed or loaded, the buttons 'Select FBS Project' and 'Select FBS Workspace' are disabled.

Test script execution:
If CODESYS File-Based Storage is not installed or loaded, loading a file-based storage project will fail.
If the path of file-based storage workspace file is used, the workspace file has to reference exactly one project. Zero or multiple referenced projects will cause the test action to fail.
TM-1775 Bug

IEC Unittest: Report an error if a testcase is not executed

Won't Fix [[GENERAL]]
Currently, this issue cannot be handled by the IecUnitTest as it is caused by a depended library.
As workaround TM-1961 will address an early evaluation of loaded projects.
TM-1701 Bug

Security issues in System.Data.SQLite versions before 2.0.4 containing SourceGear.sqlite3 3.53.4

Cannot Reproduce [[GENERAL]]
CODESYS Test Manager 5.2.0.0 does not use System.Data.SQLite anymore. Therefore this security issue is not applicable anymore -> Cannot Reproduce
TM-1673 Improvement

Replace Test Manager for Automation Platform with Opt-In

Fixed [[GENERAL]]
Legacy test drivers can now be activated using a new Test Manager option. The package CODESYS Test Manager for Automation Platform is no longer required. Users are discouraged to use these legacy test drivers unless needed, as they are not regularly maintained, localized and are not covered in our online help. The normal test drivers cover all functionality of the legacy test drivers.
TM-1658 Improvement

Add Library Parameters to Test Report

Fixed
TM-1657 Improvement

Set Library Parameter with Testmanager

Fixed
TM-1614 Bug

Typo in "license missing" message

Fixed
TM-1608 Bug

Calls to scripts with "." name prefix not considered in some commands

Fixed
TM-1600 Bug

TestManager: Complete error message not shown when halted

Fixed
TM-1587 Improvement

Test script editor: Add a "details" column to the test script tree

Fixed
TM-1562 Bug

Improve error message issued by test driver

Fixed [[COMPATIBILITY_INFORMATION]]
The test action "TestManager.Projects.LoadProject" no longer adds a fallback message handler for handling "OneOrMoreLibrariesFailedToLoad", "LossOfDataWarning" and "LossOfDataWarning2" prompts, because this message handler caused side effects, like ugly error messages and hidden storage format changes during the ongoing script execution.
TM-1557 Bug

Strange behaviour when editing ExecuteIronPythonScript's configuration settings

Fixed
TM-1539 Improvement

Remove outdated and redundant CHM-based Online Help from AddOn package

Fixed
TM-1538 Bug

Security issues in System.Data.SQLite.Core versions before 1.0.119.0 containing sqlite 3.46.1

Fixed
TM-1527 Bug

ExecuteIronPythonScript: Import fails for unknown reason

Won't Fix [[GENERAL]]
Won't fix:
Error is caused by a "wrong" script execution environment.
Both additional import paths contain a module with name that is imported in the actual test script.
The standard import logic of IronPython imports only the first module with matching name, any other module with a matching name is ignored. Therefore the behavior is as designed.

Possible solutions are:
* fix order of imports
* renaming of one of the modules
* writing a custom namespace package, which allows to have multiple modules with the same name in different locations, but might be tricky as one might not always no all modules with the given name
TM-1515 Bug

Command "Execute Test Script with Calibration" should not be added for TM "End User"

Fixed
TM-1503 Bug

TestDriverObjects: Folders are merged instead of renamed on CopyObjects

Fixed [[COMPATIBILITY_INFORMATION]]
TestDriverObjects.CopyObjects now matches the behavior of the UI and standard copy and paste commands regarding folder objects.
Folders with the same are name no longer merged, instead they are renamed / their names are deduplicated on paste.
TM-1502 Bug

TestManager: Editor for testscript can be opend twice

Fixed
TM-1479 Bug

Debugging: Out-commented test case executed, if also breakpoint is set

Fixed
TM-1471 Bug

Testtable in a library does not create the expected testcases

Fixed [[GENERAL]]
[[KNOWN_LIMITATIONS]]
* Existing TestTables within libraries still cannot be executed successfully if the libraries namespace is missing in the "testee" field
* These TestTables have to be adapted by adding the library's default namespace to the testee. The input assistant is changed accordingly and will add the default namespace
automatically, if available.
* The testtable author is responsible to check, wether the testee is correctly fully qualified
* If an alias is used instead of the default namespace, the testee type might not be found.
TM-1466 Bug

select button for Action OpenEditor is missing

Cannot Reproduce
TM-1464 Bug

[IecUnitTest] Assertion error message is missing in report

Fixed
TM-1458 Bug

TestManager: Edit Repository Dialog sometimes opens on wrong screen

Fixed
TM-1456 Bug

Error messages are not shown within IEC Unit Test report

Cannot Reproduce
TM-1454 Bug

Test checks for expected exception even if condition evaluates to False

Fixed
TM-1447 Improvement

Delete-Key and multiselect don't work in the Parameters tab of Test action

Fixed
TM-1444 Bug

TM: Objects below folder objects are inserted toplevel on cut/paste

Fixed [[GENERAL]]
The actions "TestManager.Objects.PasteSelectedObjects" and "TestManager.Objects.CopyObjects" have now the possibility to copy to a structured view as new parent.
The currently supported structured views are identified by a placeholders:
* <Devices> for the devices structured view
* <POUs> for the pool structed view
The placeholders are not case-sensitiv. The test action's configuration page supports users with buttons to add the placeholders.
TM-1441 Bug

[CI/CD-Support] TestManager: --noUI mode stops without any message

Duplicate [[GENERAL]]
This issue is duplicated by TM-129.
TM-129 introduced a new CLI command which is compatible with the '--noUI' switch.

[[KNOWN_LIMITATIONS]]
The legacy CLI command '--executecommand="TestManager ExecuteScript"' remains incompatible with the '--noUI' switch.
The new CLI command only resolves the basic no UI test script execution. It is not ensured, that all test actions are compatible with the no UI mode of CODESYS.
TM-1406 Bug

ExecuteExternalCommand: 'Save stdout to' is not executed if exit code is set

Fixed [[GENERAL]]
There is a new checkbox "Evaluate output parameter even on failure" in the Extended Settings. It always evaluates the output paramater after running the test script.
TM-1405 Bug

CloseEditor: path to non-existant object causes unhandled exception

Fixed
TM-1396 Epic

Support identifiers with special characters escaped with backticks

Fixed
TM-1368 Bug

Action "SetCommunicationPath (TestManager.Devices)" selection "Restrict to device type" does not work

Fixed
TM-1366 Improvement

IECUnitTest does not offer a possibility to work with variables for libraries to test

Fixed [[GENERAL]]
The configuration page of the IecUnitTest was changed. For the selected libraries, there is now a multiline textbox Instead of the previous tree table view. This allows full support of TestManager placeholders and generic library selection. Multiple libraries can be selected by either using a semicolon or a new line as separator. These separators can be mixed.

The selected libraries have to match one of the following three patterns:
* Placeholder library: "#LibName"
** 'LibName' can be replaced by a TM placeholder
* Library name: "LibName, LibVersion (Company)"
** 'LibName', 'LibVersion' and 'Company' can be replaced by a TM placeholder
** for 'LibVersion' the wildcard character '*' can be used
* TestManager placeholder "{variable}"
** the value of 'variable' can contain multiple libraries separated by semicolons or newlines
** each library has to match either the pattern of a placeholder library or the pattern of a library name


[[COMPATIBILITY_INFORMATION]]
If an IecUnitTest element is opened with a TestManager version greater or equal 5.2.0.0 and saved, the serialization of the IecUnitTest element may change. As long as the selected libraries match either the pattern of a placeholder library oder the pattern of a library name, the script should remain backward compatible. If one inserts a TestManager variable as placeholder which contains the whole library name, the script is no longer backward compatible.
TM-1358 Improvement

Improve "Reindex" command for unattended use

Fixed
TM-1347 Bug

TM, UnitTests: Edit-field for Selected / Excluded IEC Unittest is very small

Fixed
TM-1343 Bug

Saving a renamed Script from "-" to "_" results in unexpected error

Fixed
TM-1332 Improvement

Testmanager, Parameters: The input/output parametes should always be trimmed

Cannot Reproduce [[GENERAL]]
Cannot reproduce: parameters are already trimmed
TM-1324 Bug

Variables in Input and Output Parameters should be trimmed automatically

Fixed
TM-1294 Bug

if an action has expected exception, a previous failed action didn't skip the action

Fixed
TM-1286 Improvement

Update reports view to WebView2

Fixed
TM-1279 Bug

CheckDeviceState is not working as expected.

Won't Fix [[GENERAL]]
Logic of CheckDeviceState can not be adjusted to handle the reported issue correctly, because of compatability reasons. To handle the error properly a new test action would be required which still has to be specified.
TM-1150 Bug

repositories are not usable in the seconde instance

Duplicate [[GENERAL]]
This issue was already addressed with the reworked test script repository (TM-1619).
TM-1133 Bug

ChangeDeclaration: Error when there is an attribute

Cannot Reproduce
TM-1129 Bug

CompareVariableList: Not possible to use Parameter

Cannot Reproduce
TM-1018 Improvement

Possbility to abort the running subscript

Fixed [[GENERAL]]
- Checkbox 'Cancel Call on Failure' has been added in Extended Settings of action
- It's mutually exclusive to other checkboxes in Extended Settings
- If the action with 'Cancel Call on Failure' fails, all following actions and test cases in a Call will be skipped
- Setting 'Cancel Call on Failure' checkbox overwrites 'Execute On Failure' checkbox
TM-1010 Bug

Example tests: ApplicationTest project not compileable anymore

Won't Fix [[GENERAL]]
The examples for the CODESYS Test Manager are moved to a dedicated examples package (EX-322).
TM-1008 Bug

A commented-out Call is still active

Fixed [[GENERAL]]
This behavior is as designed and it is also well documented in the online help: https://content.helpme-codesys.com/en/CODESYS%20Test%20Manager/_tm_cmd_comment_uncomment.html
TM-913 Bug

TestManagerEditor: Potential Conflict State of a Testscript due second TestScriptEditor View

Duplicate
TM-908 Bug

Report AddLibraryVersionInformation with not installed libs is not possible

Cannot Reproduce [[GENERAL]]
Cannot reproduce
* test action "AddLibraryVersionInformation" succeeds even if a library is not installed
* for the non-installed library "Version: Library is not installed" is reported
TM-882 Improvement

Test Manager: Signal if execution is halted

Fixed
TM-756 Bug

The result of a subordinate script should be reflected in the result of the calling script.

Fixed
TM-743 Bug

TestManager: Test progress window is not visible any more

Duplicate [[GENERAL]]
Duplicates TM-748
TM-585 Improvement

[CI/CD support] TestManager: Display error if path to report file is not valid

Cannot Reproduce
TM-576 Bug

IEC Unit Test: Multitests do not work in the simulation mode

Fixed
TM-510 Bug

TestDriverMonitoring: CheckIncrement: BIT is not selectable

Fixed
TM-472 Bug

TestTable: Simulation mode does not work

Cannot Reproduce
TM-419 Improvement

Check Library Parameter with Testmanager

Fixed
TM-396 Improvement

IECUnitTest: Increase length of sInfo string

Cannot Reproduce
TM-283 Improvement

TestDriverIecUnitTest: Allow configuration of the timeout

Won't Fix [[GENERAL]]
Won't fix: a global overwrite of the testcase timeout might lead to problems on unittest execution, especially if the unittest are defined in libraries and the tester has no possibility to add/modify the "testcasetimeout" attribute.
Additionally, the "testcasetimeout" attribut is meanwhile well explained, see https://content.helpme-codesys.com/en/CODESYS%20Test%20Manager/_tm_pragmas_in_test_blocks.html
TM-129 Epic

[CI/CD support] TestManager: Enable --noUI mode

Fixed [[GENERAL]]
CODESYS Test Manager now supports running tests with the "--noUI" command line switch. This requires the use of Test Manager's new CLI command "--runtest". The legacy CLI command, that used "--executecommand" does not support it. This feature requires at least CODESYS V3.5 SP22. See the online help for details on the new CLI. Licensing is designed so, that a test run is never aborted, if the situation changes from "licensed" to "unlicensed" (e.g. because of network license problems), but that without license no test report will be created (with and witout UI).

[[COMPATIBILITY_INFORMATION]]
Remote Calls will fail, if the target installation used for the remote call has no CODESYS Test Manager 5.2.0.0 or newer installed. If neither the remote call configuration 'Path of remote executable' nor 'Installation name of remote instance' is used, there will be no issue regarding remote call compatibility.

[[KNOWN_LIMITATIONS]]
Running tests in "--noUI" mode requires at least CODESYS V3.5 SP22. Depending on the CODESYS version modal dialogs might still pop up and block the test run. This can be handled using the new "--strict" CLI switch introduced by CDS-99384 (CODESYS V3.5 SP23).