Invoking Python script from Java

Python scripts could be invoked from Java using different frameworks. We have done analysis on the below frameworks and finally selected Java Process Builder framework for the final design and implementation. All the analysis findings of different frameworks are as follows.

Approaches Considered

Jython

Jython is an implementation of the Python programming language designed to run on the Java platform. Adding Jython on the CLASSPATH, the framework should automatically discover that we have the possibility of using this scripting engine and enable us to ask for the Python script engine directly. The below are the advantages and drawbacks of using Jython

Advantages

1. It provide mechanisms for executing scripts and for sharing values between Java and python
2. Jython brings the possibility of embedding Python code directly into our Java code. We can do this using the PythonInterpretor class
3. Jython compiles Python source code to Java bytecode (an intermediate language) either on demand or statically.

Drawbacks

1. Jython only support python scripts written using python 2, it will support only till python 2.7.x
2. Jython is the Python implementation for Java, it may not contain all the same sub-packages as native Python
3. Adding python dependencies to the native python package in the VM wont be reflected in Jython's local repository causing dependency not available error during python script execution

Java Process Builder

The ProcessBuilder API can be used to create a native operating system process to launch python. We can create our ProcessBuilder object passing the command and argument values to the constructor. It's also important to mention the call to redirectErrorStream(true). In case of any errors, the error output will be merged with the standard output. 

1. No dependencies needs to added to CLASSPATH
2. Python should be installed in the VM in which the python script invocation happens
3. Incase both python 3 and python 2 installed in the VM there will be a discrepancy in identifying the python command( eg: python and python3) to be used and python version to be used for executing the script.

Java Embedded Python (JEP)

Jep stands for Java embedded Python. It is a mirror image of JPype. Rather that focusing on accessing Java from within Python, this project is geared towards allowing Java to access Python as sub-interpreter. The syntax for accessing Java resources from within the embedded Python is quite similar with support for imports. Jep embeds CPython in Java through JNI.

Some benefits of embedding JEP in a JVM:

• Using the native Python interpreter may be much faster than alternatives.
• Python is mature, well supported, and well documented.
• Access to high quality Python modules, both native CPython extensions and Python-based.
• Compilers and assorted Python tools are as mature as the language.
• Python is an interpreted language, enabling scripting of established Java code without requiring recompilation.
• Both Java and Python are cross platform, enabling deployment to different operating systems.

Dependencies for using JEP to execute python scripts :

• Python 3.5 or above should be installed in the VM in which Automatics python test case execution happens
• The jep.jar is accessible to the Java classloaders (typically through the Java classpath)
• The shared library (jep.so or jep.dll) is accessible by the Java process (typically through -Djava.library.path or the environment variable LD_LIBRARY_PATH)
• The jep python files (console.py, java_import_hook.py, version.py, etc.) are accessible by Python (typically by placing them in the site-packages/jep directory).

Refer this link for JEP installation and setup documentation : https://github.com/ninia/jep/wiki/Getting-Started

HTTP

Using HTTP server Python projects needs to be started as a REST api server using the Python frameworks like flask or Django. If REST api are created similar to the API template provided automatics, then these API's can be invoked from automatics and there by executing the python test cases and updating the response back to automatics.

Invoking Java class from Python Script

Approaches Considered

JPype

Official Documentation : - https://jpype.readthedocs.io/en/latest/

JPype is a Python module to provide full access to Java from within Python. Unlike Jython, JPype does not achieve this by re-implementing Python, but instead by interfacing both virtual machines at the native level. This shared memory based approach achieves good computing performance, while providing the access to the entirety of CPython and Java libraries. This approach allows direct memory access between the two machines, implementation of Java interfaces in Python, and even use of Java threading. JPype user guide is available in this link : https://github.com/jpype-project/jpype/blob/master/doc/userguide.rst

JPype is designed to allow the user to exercise Java as fluidly as possible from within Python. We can break this down into a few specific design goals.

• Make Java appear Pythonic. Make it so a Python programmer feels comfortable making use of Java concepts. This means making use of Python concepts to create very Python looking code and at times bending Python concepts to conform to Java's expectations.
• Make Python appear like Java. Present concepts from Java with a syntax that resembles Java so that Java users can work with Python without a huge learning curve.
• Present everything that Java has to offer to Python. Every library, package, and Java feature if possible should be accessible. The goal of bridge is to open up places and not to restrict flow.
• Keep the design as simple as possible. Mixing languages is already complex enough so don't required the user to learn a huge arsenal of unique methods. Instead keep it simple with well defined rules and reuse these concepts. For example, all array types originate from JArray, and thus using one can also use is instance to check if a class is an array type. Rather than introducing factory that does a similar job to an existing one, instead use a keyword argument on the current factory.
• Favor clarity over performance. This doesn't mean not trying to optimize paths, but just as premature optimization is the bane of programmers, requiring writing to maximize speed is a poor long term choice, especially in a language such as Python were weak typing can promote bit rot.
• If a new method has to be introduced, make look familiar. Java programmers look to a method named "of" to convert to a type on factories such as a Stream, thus JArray.of converts a Python NumPy array to Java. Python programmers expect that memory backed objects can be converted into bytes for rapid transfer using a memory view, thus memoryview(array) will perform that task.
• Provide an obvious way for both Python and Java programmers to perform tasks. On this front JPype and Python disagree. In Python's philosophy there should be one -- and preferably only one -- obvious way to do things. But we are bridging two worlds and thus obviousness is in the eye of the beholder.

Py4j

Py4J is a library that enables Python programs to dynamically access Java objects in a Java Virtual Machine (JVM). It allows Python code to interact with Java code and vice versa, making it useful for integrating Python and Java programs. Py4J includes a gateway that connects the Python and Java programs, and allows for the exchange of data and calls between the two languages. 

Py4J uses sockets to communicate between the Java and Python programs. When a Py4J gateway is created, it starts a server socket that listens for incoming connections from the Python program. The Python program can then connect to this socket and send commands to the Java program via the socket connection.

The Java program, in turn, sends the results of these commands back to the Python program via the same socket connection. This allows for a two-way communication between the Java and Python programs.

Py4J uses the socket connection to pass data and method calls between the two languages, and it takes care of the details of serializing and deserializing the data, so that the two programs can understand each other.

Since it uses sockets to communicate parallel execution in same vm will be difficult