Our security department requires us to use log4j 2.17 which is embedded in SoapUI 5.7.0. This requires us to upgrade our current version of SoapUI v5.2.1.
I tested few users scripts with SoapUI 5.7.0 and all run smoothly but the few integration tests fail. I’ve been searching for days without finding the solution to this issue. The issue is not related to our code since command testrunner.sh fails with the exact same parameters (see below).
I first run the mockservicerunner:
ubuntu@tab-sabrina-dev:~/MyRepo/git/TSM/hestia-software-scripts$ mockservicerunner.sh -m"REST MockService" -p 58475 test/integration/monitoring/soapui/soapui-project.xml
================================
=
= SOAPUI_HOME = /home/ubuntu/SmartBear/SoapUI-5.7.0
=
================================
OpenJDK 64-Bit Server VM warning: Option --illegal-access is deprecated and will be removed in a future release.
SoapUI 5.7.0 MockService Runner
17:40:40,385 INFO [DefaultSoapUICore] Creating new settings at [/home/ubuntu/soapui-settings.xml]
17:40:45,326 INFO [PluginManager] Adding plugin from [/home/ubuntu/.soapuios/plugins/ready-mqtt-plugin-dist.jar]
17:40:45,334 INFO [PluginManager] Adding plugin from [/home/ubuntu/.soapuios/plugins/readyapi-postman-1.0.1.jar]
17:40:46,571 INFO [PluginManager] Adding plugin from [/home/ubuntu/.soapuios/plugins/ready-uxm-plugin-os-1.0.3.jar]
17:40:46,991 INFO [PluginManager] 3 plugins loaded in 1661 ms
17:40:46,992 INFO [DefaultSoapUICore] All plugins loaded
17:40:50,191 INFO [WsdlProject] Loaded project from [file:/home/ubuntu/MyRepo/git/TSM/hestia-software-scripts/test/integration/monitoring/soapui/soapui-project.xml]
17:40:50,201 INFO [SoapUIMockServiceRunner] Running MockService [REST MockService] in project [Hestia Integration]
17:40:50,201 INFO [SoapUIMockServiceRunner] Press any key to terminate
17:40:50,797 INFO [JettyMockEngine] Started mockService [REST MockService] on port [58475] at path [/]
17:40:50,803 INFO [SoapUIMockServiceRunner] MockService started on port 58475 at path [/]
17:40:50,803 INFO [SoapUIMockServiceRunner] Started 1 runner
Press any key to terminate...
17:43:09,496 INFO [log] Path: /pet/findByStatus
17:43:09,623 INFO [log] QueryString: status=available
17:43:09,707 INFO [SoapUIMockServiceRunner] Handled request 1; [/pet/findByStatus] with [Available] in [0ms] at [2022-06-28 17:43:09.523]
17:43:10,732 INFO [log] Path: /pet/findByStatus
17:43:10,733 INFO [log] QueryString: status=available
17:43:10,749 INFO [SoapUIMockServiceRunner] Handled request 2; [/pet/findByStatus] with [Available] in [0ms] at [2022-06-28 17:43:10.733]
17:43:10,770 INFO [log] Path: /pet/findByStatus
17:43:10,771 INFO [log] QueryString: status=available
17:43:10,790 INFO [SoapUIMockServiceRunner] Handled request 3; [/pet/findByStatus] with [Available] in [0ms] at [2022-06-28 17:43:10.770]
17:43:10,914 INFO [log] Path: /pet/findByStatus
17:43:10,917 INFO [log] QueryString: status=available
17:43:10,931 INFO [SoapUIMockServiceRunner] Handled request 4; [/pet/findByStatus] with [Available] in [0ms] at [2022-06-28 17:43:10.915]
17:43:10,950 INFO [log] Path: /pet/findByStatus
17:43:10,951 INFO [log] QueryString: status=available
17:43:10,963 INFO [SoapUIMockServiceRunner] Handled request 5; [/pet/findByStatus] with [Available] in [0ms] at [2022-06-28 17:43:10.951]Then I run the test with testrunner:
$ testrunner.sh -rAI -Duser.timezone=UTC -Dlogfilename=TestREST_-_test_simple_success_latest.log -Derrfilename=TestREST_-_test_simple_success_errors.log -Dsoapui.log.dir=test/integration/monitoring/soapui/rest/simple/success/logs -s 'Simple - REST Only - success' -h localhost:58475 -e http://localhost:58475 test/integration/monitoring/soapui/rest/simple/success/project.xml
================================
=
= SOAPUI_HOME = /home/ubuntu/SmartBear/SoapUI-5.7.0
=
================================
<snip>
17:59:41,810 INFO [DefaultSoapUICore] All plugins loaded
17:59:43,215 INFO [WsdlProject] Loaded project from [file:/home/ubuntu/MyRepo/git/TSM/hestia-software-scripts/test/integration/monitoring/soapui/rest/simple/success/project.xml]
17:59:43,219 INFO [SoapUITestCaseRunner] Running SoapUI tests in project [Hestia Integration]
17:59:43,220 INFO [SoapUITestCaseRunner] Running TestSuite [Simple - REST Only - success], runType = SEQUENTIAL
17:59:43,226 INFO [SoapUITestCaseRunner] Running SoapUI testcase [No assertions]
17:59:43,230 INFO [SoapUITestCaseRunner] running step [Get available pets]
17:59:44,245 INFO [SoapUITestCaseRunner] Finished running SoapUI testcase [No assertions], time taken: 731ms, status: FINISHED
17:59:44,254 INFO [SoapUITestCaseRunner] Running SoapUI testcase [Status code assertion]
17:59:44,254 INFO [SoapUITestCaseRunner] running step [Get available pets]
17:59:44,285 INFO [SoapUITestCaseRunner] Assertion [Valid HTTP Status Codes] has status VALID
17:59:44,290 INFO [SoapUITestCaseRunner] Finished running SoapUI testcase [Status code assertion], time taken: 12ms, status: FINISHED
17:59:44,290 INFO [SoapUITestCaseRunner] Running SoapUI testcase [JSON path existence assertion]
17:59:44,291 INFO [SoapUITestCaseRunner] running step [Get available pets]
17:59:44,346 INFO [SoapUITestCaseRunner] Assertion [JsonPath Existence Match - Id is returned] has status VALID
17:59:44,347 INFO [SoapUITestCaseRunner] Finished running SoapUI testcase [JSON path existence assertion], time taken: 2ms, status: FINISHED
17:59:44,350 INFO [SoapUITestCaseRunner] Running SoapUI testcase [JSON path match assertion]
17:59:44,350 INFO [SoapUITestCaseRunner] running step [Get available pets]
17:59:44,372 INFO [SoapUITestCaseRunner] Assertion [JsonPath Match - First status is available] has status FAILED
17:59:44,372 ERROR [SoapUITestCaseRunner] ASSERTION FAILED -> Comparison failed for path [$..status[0]], expecting [available], actual was [[]]
17:59:44,372 ERROR [SoapUITestCaseRunner] ASSERTION FAILED -> Comparison failed for path [$..status[0]], expecting [available], actual was [[]]
17:59:44,373 ERROR [SoapUITestCaseRunner] Get available pets failed, exporting to [/home/ubuntu/MyRepo/git/TSM/hestia-software-scripts/Simple__REST_Only__success/JSON_path_match_assertion/Get_available_pets-0-FAILED.txt]
17:59:44,373 ERROR [SoapUITestCaseRunner] Get available pets failed, exporting to [/home/ubuntu/MyRepo/git/TSM/hestia-software-scripts/Simple__REST_Only__success/JSON_path_match_assertion/Get_available_pets-0-FAILED.txt]
17:59:44,373 INFO [SoapUITestCaseRunner] Finished running SoapUI testcase [JSON path match assertion], time taken: 3ms, status: FAILED
17:59:44,374 INFO [SoapUITestCaseRunner] Running SoapUI testcase [JSON path regex match assertion]
17:59:44,374 INFO [SoapUITestCaseRunner] running step [Get available pets]
17:59:44,399 INFO [SoapUITestCaseRunner] Assertion [JsonPath RegEx Match - Pet name is valid] has status FAILED
17:59:44,400 ERROR [SoapUITestCaseRunner] ASSERTION FAILED -> Comparison failed for path [$..name[0]], expecting [true], actual was [false]
17:59:44,400 ERROR [SoapUITestCaseRunner] ASSERTION FAILED -> Comparison failed for path [$..name[0]], expecting [true], actual was [false]
17:59:44,400 ERROR [SoapUITestCaseRunner] Get available pets failed, exporting to [/home/ubuntu/MyRepo/git/TSM/hestia-software-scripts/Simple__REST_Only__success/JSON_path_regex_match_assertion/Get_available_pets-0-FAILED.txt]
17:59:44,400 ERROR [SoapUITestCaseRunner] Get available pets failed, exporting to [/home/ubuntu/MyRepo/git/TSM/hestia-software-scripts/Simple__REST_Only__success/JSON_path_regex_match_assertion/Get_available_pets-0-FAILED.txt]
17:59:44,402 INFO [SoapUITestCaseRunner] Finished running SoapUI testcase [JSON path regex match assertion], time taken: 2ms, status: FAILED
17:59:44,405 INFO [SoapUITestCaseRunner] TestSuite [Simple - REST Only - success] finished with status [FAILED] in 1179ms
SoapUI 5.7.0 TestCaseRunner Summary
-----------------------------
Time Taken: 1185ms
Total TestSuites: 1
Total TestCases: 5 (2 failed)
Total TestSteps: 5
Total Request Assertions: 4
Total Failed Assertions: 2
Total Exported Results: 5
I know that there’s a bug with JsonPath Match assertions in SoapUI 6.7.0 (see question #53054188) and replaced them with XPath Match but it still doesn’t work and the same error message appears.
Any help would be greatly appreciated.
Best regards,
Sabrina
2
Answers
I finally figured out what the pb was: this is most likely related to the upgrade of the Jayway jsonpath library from 0.9.1 (SOAPUI 5.6) to 2.4.0 (SOAPUI 5.7). See SOAPUI release notes:
In the project file, replacing "$..status[0]" with " $[0].status" solved the issue.
I used this website for JSONPath implementation differences and validation: https://jsonpath.herokuapp.com/.
SoapUI 5.7.0 has upgraded a JSONPath library (Jayway).
In the previous version, Jayway JSONPath allowed the bracket [] operator on any JSON node. In the recent version, this behavior has been "fixed" so the bracket operator [] can be applied only to JSON arrays.
Here’s more about the JSONPath selection issue.