Category Archives: jython

Blast from the past: Taming the Croatian eHealth Beast

Reading Time: 2 minutes

TL;DR

 

We would like to describe and explain how session GTAC 2008: Taming the Beast – How to Test an AJAX Application had direct impact on refactoring our testing strategy for E-Health Croatia (eHealthCRO).

 

Our system has been implemented using J2EE platform. It the first instance of test plan, test team was responsible for designing and implementing end-to-end test scripts for every test case from test plan. Soon, we realized that regression test was taking too long, and we needed faster development-test-deployment cycle. Using tips and knowledge from the mentioned session, we deployed suggested testing strategy on our system.

 

In the presentation we will explain challenges that we had met during this process, along with concrete technical solutions in order to implement proposed testing techniques.

As an introduction we will give architecture overview of eHealthCRO system. System consists of several applications packaged as J2EE 1.4 enterprise application.

User has to compose appropriate Health Level 7 v3 standard xml message in order to complete end-to-end system flow for appropriate eHealthCRO service. We will give an example of sending ePrescription in order to explain end-to-end system flow and interfaces through which subsystem communicate with each other.

 

User creates HL7 xml message with prescription data as payload. First subsystem is Gate. Gate is responsible for:

 

  • HL7 xml message syntax checking according to appropriate schema

    xml digital signature checking

  • user authentication and authorization

  • forwarding sync request to central Presc application (using ejb 2.1 and IIOP as protocol)

    forwarding response back to user.

  • Gate application is running on its own Sun Java System Application Server Enterprise Edition 8.2 application server instance.

Business application:

  • implements prescription business logic using Spring beans

  • using Hibernate ORM communicates with Informix 11 relational database.

  • it runs on its own instance of Sun Java System Application Server Enterprise Edition 8.2 application server instance

 

Most of test cases from test plan where about testing prescription business logic. In order to speed up test execution, we developed ejb 2.1 client that calls business logic methods same as one module from Gate application. Developing that client was very easy and fast because we used jython 2.5 programming language. Client code was already developed in Java by development team, so testers only created jython wrapper methods around Java code.
Doing this, our test suite was 20 times faster than before. We received information about implemented business logic much faster than using end-to-end system testing.

 

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

jython and pip ‘No files/directores in pip-egg-info’ issue

Reading Time: 2 minutes

My goal is to drive Sikuli using plain-text functional descriptions, like Cucumber does for a Ruby. Sikuli uses Jython, so my first step was to install Lettuce Python module (Cucumber for Python). And everything must run on Windows. I am using Jython2.7a2, and first step was to install pip on Jython. Here is stackoverflow answer for installing pip in Jython. Second step was installation of Lettuce module, and simple:

>jython %JYTHON_HOME%binpip lettuce

does not work.
Following command:

>jython %JYTHON_HOME%binpip –proxy my_proxy_username:@proxy_url:port -r requirements.txt

produced following error (yes, I am working behind proxy server, and it was real fun to find out proxy information from IT in my company):


“No files/directores in %your_windows_home_directory%locals~1temppip-buildJinja2pip-egg-info (from PKG-INFO)
Storing complete log in %your_windows_home_directory%.pippip.log”

and Google told me that this is known Jython issue. Notice that issue has English grammatical error.
I resolved that issue with following command:

>cd  %your_windows_home_directory%Local SettingsTemppip-buildJinja2
>jython setup.py install

What I actually did? pip has two steps. First is to get the module egg from pip repository, and second is to run the actual install of the egg. I run second step by hand.
You will have to do that for all modules listed in requirements.txt, and when you install one module, you have to delete it from requirements.txt file.

Final step, because my test machine does not have access to Internet, is to copy/paste JYTHON_HOME folder from my development machine (where all pip installations are done) to the test machine.

Update 01:

Update1:
During the installation of Pygments module (jython setup.py install) I received following error:
“java.lang.RuntimeException: java.lang.RuntimeException: Method code too large!”
More on that you can read again in Jython issue 1891:

I successfully installed Pygments after I divided definition of three lists in file:

%my_windows_home%Local SettingsTemppip-buildPygmentspygmentslexers_vimbuiltins.py

into three files in the same folder as original file:
_vimbuiltins.py
_vimbuiltins_options.py
_vimbuiltins_command.py






Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

Testing experience with utf-8

Reading Time: 3 minutes

I would like to describe what we have learned while testing with utf-8 encoded Croatian characters.

Introduction.

One of our previous project issue was java virtual machine (jvm) thread hang. Web service servlet started application server thread by processing incoming xml message. Xml encoding is utf-8.
     
Then, using spring and hibernate frameworks, jdbc connection towards the Informix database was used in order to store the data into the database. And that thread hang infinitely in some cases in production environment. After we gathered customer bug reports, we confirmed that xml with ‘broken’ encoding caused the hang. By broken encoding we mean when some Croatian character (e.g. ‘š’ with hex code c5 a1) was encoded with some other hex code. We reproduced the case by copy/pasting part of xml message with broken utf-8 Croatian characters. That xml was in mantis bug report. At that moment we did not know how to produce those broken encoding characters. At server side we implemented the code that intercepted those broken encoding and returns appropriate error.

Problem

During the regression test, using the Python as a tool for automation, tester @majapenovic received the broken encoding error. She asked another tester that wrote the testing script about the problem and his solution was to delete Croatian characters used for generating the xml input message. This is VERY BAD TESTERS DECISSION. I told Maja to investigate what was the root of the problem. 

utf-8 and Python

We learned about how to use utf-8 in Python from this excellent post.
We are using Jython for writing integration scripts. So from bottom up, you should configure following for proper utf-8 string manipulation:

  • jvm that runs jython should have following java option: -Dfile.encoding=UTF-8 (You can find that option in bin/jython file.)
  • At the begining of jython script: #coding=utf-8
  • your editor encoding must be set to utf-8.
  • your keyboard must be set to Croatian (if you work with Croatian utf-8 character set)
Your are know ready to write following code:

croCharsAsByteStream = “ŠĐČĆŽšđčćž”

croCharsInUnicodeUTF8

 = unicode (croChars,”utf-8″)

# put back in byte string, for print function, http transport

croCharsBackInByteStream= croCharsInUnicode.encode(‘utf-8’)
croCharsInUnicodeUTF8 = byteStreamReceivedFromHttp.decode(‘utf-8’)
#compare in verification check byte stream with byte stream!


For writing byte stream to file we use following code:

f = codecs.open (file_path, ‘w’, “utf-8”)
f.write(croCharsAsByteStream)

I helped Maja to check is our xml message created with broken encoding. Every xml message is stored in the database. Database encoding was set to utf-8. We unloaded the record with our xml message (informix unload statement), and used vi in hex view mode (:%!xxd) to observe the hex encoding for the character ‘š’ in xml message. At we confirmed that it had wrong hex values.

Tester magic

Maja started investigation. She found bug in our testing script. We called encode() method on byte stream twice in a row in different methods. That caused broken encoding of xml message.
Maja did not followed any best practice, she adopted to context of the problem. She used the existing code functionality. Write to file was called in several places, and she decided to observe the output of that methods. At some point, the output had broken encoding for character ‘š’. After that she easily spotted the line with bug in our script. 


What we learned

Testers job is to find proper solutions, not to do ‘dirty workarounds’. As a result of problem investigation, we now know how to reproduce the broken encoding error.

Question


Are you able to see in browser Croatian characters from this blog post?

Update regarding replace string method

If you need to do something like this:

request = request.replace(“placeholder”, unicode( ‘Š’, ‘utf-8’ ) )

you will get following exception:

UnicodeDecodeError: ‘ascii’ codec can’t decode byte

only if string in which you are doing the replacement (in this example request string) is not in utf-8 format. You usually put some string in utf-8 either by using decode(‘utf-8’) method, or reading a file using following code snippet:


f = codecs.open (‘file_path’, ‘r’, “utf-8”)
request= f.read()

Update2 regarding UnicodeDecodeError: ‘ascii’ codec can’t decode byte

Today I found an excellent post about famous exception


UnicodeDecodeError: ‘ascii’ codec can’t decode byte

From now I finally understand that rather strange exception. In Python 2.x encode and decode methods work on unicode strings. That means if you try to call them on byte object, Python will implicitly try to decode  byte to unicode object using default ‘ascii’ encoding. If your byte object contains bytes out of ascii range, you will trigger that exception.
The problem is that Python2.x is using ascii as default encoding because of the “historical reasons”.  

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather