Monday, April 5, 2010

Switching from C# to Java

In 2008 RedHat acquired Qumranet, a startup whose focus was Virtualization. Among other products Qumranet developed a management application for Virtualization.

The management application was written in C# and one of the first tasks we got was to make the management application cross platform, well this was expected considering the fact that the acquisition was done by RedHat...

We started exploring the web looking for ideas how to approach this task. At the beginning things did not look promising most of the references we found for porting projects from one technology to another were about complete failures, the only obvious suggestion that we saw all over was not to change technology and architecture at the same time.

Armed with this important advice we kept digging around and realized there are two different paths we can take. The first was to stick with C#, and the second, surprisingly, was to change technology.

Sticking with C# requires technologies to help us run it on Linux. We found Mono which is an open source implementation of Microsoft's .NET Framework which can be executed on Linux, and the second option we found was Grasshopper which is a project of Mainsoft to compile MSIL to Java Bytecode. The idea is to code in C#, compile it to Java Byte code and run the code on JRE on any JRE enabled platform, Linux included. Cool stuff right?
These 2 solutions were taken off the table, we are on our way to open source our project (Red Hat remember...) and we wanted to use a technology supported by Red Hat. In addition a basic POC we did (POC #1) using MONO proved that the technology was immature at the time and did not meet our needs.

Okay switching technology then. At this point Java looked like the natural choice. It is cross platform, has a lot of enterprise framework ready to use and it's syntax and OO principles are very similar to C# which is an advantage when it comes to training the development team.

Here is a problem for CS guys:
input: 2.5 developers, 4 month period, 100k lines of C# code and a relatively mature management application
output:parity application in java
constraint: during these 4 months 4 developers are adding required features on the C# version
algorithm: ?


After we realized we have no cheat sheet for this problem we considered 3 options:
1. Manual - Writing it all from scratch in java
2. Hybrid - integrate Java modules in the C# application
3. Automatic - automatically convert the code (Yes we believe in miracles, at this point who wouldn't?)

Manual -The obvious option was writing it from scratch, every developer's dream, who does not think that the second time he will write the same code in the most generic flawless way? well nice but no thank you, first of all it is most likely that we will not do the same mistakes again instead we will introduce new bugswhich will probably take forever to fix. In addition we will probably lose whatever maturity the application has. Another con which eventually ruled this option out was that writing from scratch in Java while trying to catch up on the moving target in C# seems impossible to us. We did another POC (POC #2) with a new generic architecture to get an estimate of the time and amount of work for this approach.


Hybrid - The current architecture of the C# version is based on the Command design pattern. Generally we can say that the flows/actions in the system can be mapped to commands. The Hybrid approach was to gradually port the flows from C# to java and during the process have both technologies live side by side. The obvious pros of such an approach are that we can have the system running at all times which can be good for the system maturity, we could also keep developing new features in that period (write them in Java of course) and we could harness the whole team for the conversion (+4 developers to work with us). Looking at the cons this is considered a high risk path, we cannot deliver such a product until the conversion process is complete and the time estimates for this process were kind of random, it actually risked the next release (which originally was planned in C# and was due at the end of the 4 month period), another con was that we realized that for the Hybrid approach we had to have some of the infrastructure ready in Java before we start migrating flows, which turned out to be a lot of code, we might as well port the whole thing. One other difficulty we encountered was that managing database transactions in such a system was very problematic. Anyway the high risk was decisive and yet another POC goes down the drain (POC #2.5, we can't actually call it a POC because it was never executed...)


Automatic - The inspiration for this was an article about Boeing and automatic conversion. Well we thought "if Boeing can do it so can we". Sounds stupid? Well it is.
Luckily for us we did not think that at the time. We looked for automatic conversion tools from C# to java, we actually came across 2 of them one is net2java, an abandoned project, which did not help us much, the second one was Tangible with which we started a POC (POC #3).
At the beginning it looked horrible, we converted the project and got over 50K compilation errors but as I mentioned the Boeing article was inspiring (plus we did not have other options) so we tried looking into the compilation errors, what we found was that some of them were repetitive and by doing a little work we can eliminate some of them.
We started by sending Tangible one or two issues which dominated the errors list, we were pleasantly surprised by Tangible's support, they were so cooperative and fast (or shall I say "he was", Dave, was one very efficient developer) it actually encouraged us to send a second chain of bugs followed by a third and a forth. We ended up filing more than 200 issues (over a 6 month period) of bigger and smaller issues we encountered on our journey to Java.

In addition to the ongoing effort with Tangible we had to dumb down the C# code.
For example we had to remove the usage of the Linq library in the C# side since it is not converted to Java properly.
And in the same time we wrote some "sed" scripts to manipulate the Java output and fix some errors like packaging, adding import statements at the beginning of the class or adding a static data member (logger) in each class.

Obviously there was some manual work as well, but I'll elaborate on the technical details of the conversion process itself in my next post.

It took us around 4 months to stabilize the process and get to a working version of our application in Java. We got to a parity version in Java which passed 90% of our automatic testing. We sometimes still can't believe it.
We did not do a scalability test on the Java version yet, I will blog about it as soon as we have some results.


Although for automobiles the next generation is hybrid for us automatic conversion did the job.

Cheers,
Livnat

39 comments:

  1. Livnat,

    Thanks for sharing this important post.

    We are also currently working on porting a legacy .NET server (Windows + IIS + WCF) to Java EE (Linux + JBoss + EJB3 + JAX-WS) and initially considered Manual and Hybrid approaches.

    We finally choose the Manual approach as we believed that re-designing the architecture is required in our case. I believe that Java has more to offer to enterprise server development than .NET (but I am sure people will argue me with me about it).

    For the Hybrid approach we also considered JNBridge which allows very good .NET <-> Java interoperability and looks very promising.

    We didn't seriously consider the Automatic approach for the same reason you mentioned (will it really work?). WELL DONE!

    One more problematic aspect you didn't mention (and I will love to hear from you) is the political one. Taking a team of (experienced) C# developers and telling them to do Java from now on, to my experience, is not trivial at all!

    Although syntax is similar between the platforms, you have all the different JSRs (EJB, JPA, JDBC, CDI, JAX-WS, JMS, JCA, Servlet, etc.) which are on top of the Java language and are not trivial to master even for experienced Java developers (not to mention new comers to Java).

    Shai

    sl1977@bezeqint.net

    ReplyDelete
  2. My advice to anyone going this way is don't pick the default stack(EJB,JSF,etc). The openness created a lot of diversity in the java platform so there are millions of frameworks and platforms to pick each with up and downsides a job that can be most daunting.

    I'm most confortable with Spring and Grails, but Wicket, Guice, GWT all can be a great fit for your problem and your team.

    ReplyDelete
  3. @Shai
    Thank you for your comment.
    I can relate to the need to re-design and fix flaws in the system, at the same time I think that one of the best decisions we made was not to change technology and design in parallel.
    I think that doing the full switch both technology and design might be too much to handle, maybe having some stepping stones on the way might ease the process a little bit.
    After you switch technology and stabilize your application (sticking to the original design) you have the same application 'only' in another language, that is easier to process mostly for developers who are new to the technology and not less important to the working environment (IDE, build tools like maven, DB etc.).
    Then as a second step you can redesign your system, maybe do it one component at a time. By this time the team is more acquainted with the technology and might feel more comfortable with changing the architecture, same goes for adopting new frameworks, you might want to start with the bare minimum and grow from there.
    Another huge advantage is that by doing one step at the time after the first step you can decide if you want to redesign your system or introduce new features to it, maybe some combination of the two. You might also add stability and maturity to your system in the process.

    About the difficulties of the team, I can say that I am lucky to work with some open minded developers, fact that really made the process more comfortable for us.
    Although to be honest we are still struggling with the change.
    Good luck with your conversion process,
    Livnat

    ReplyDelete
  4. Nice post!
    Good work! :)

    mku.

    ReplyDelete
  5. we ported 260k C++ application to Java with 2 developers in 15 months (still not fully ready for system testing). We changed both language and technology. but additional complexity was that we also changed platform. Initial estimation was 6 months. but it looks all in all it will take 20 months.

    ReplyDelete
  6. Is the current java source tree available for community consumption yet?

    ReplyDelete
  7. Had you considered incremental porting from C# to J#, which is Java syntax, perhaps adding abstractions for to be more Java library compliant (Java collections, etc.), then doing a manual port of the J# to actual Java? This seems like it could have been productive, and yet allow you to keep your system up and running and progressing.

    Perhaps rearchitecting to Spring.NET (I don't know how compatible Java Spring and Spring.NET, but I would imagine they are quite similar).

    ReplyDelete
  8. @Anonymous
    The code is not available for community consumption yet, but it is on our road map.

    @Will Hartung
    We did not consider using J# or Spring.Net as an intermediate state, interesting idea though.
    Two things that come to mind -
    1. We don't have in house knowledge of these technologies.
    2. Having another technology as a stepping stone might increase risk.

    ReplyDelete
  9. JNBridge looks like stuff that allows to connect C# to Java, which is not what was asked for here.

    ReplyDelete
  10. Why was Mono immature? We are using it heavily within our projects.

    ReplyDelete
  11. I imagine that some of your .NET code was relying on .NET libraries to talk to Active Directory domains. Did this code get automatically converted or did you have to manually convert it?
    Also did you bump into any issue with regards to Java Active Directory libraries?

    ReplyDelete
  12. Can I please ask how was testing being done on the ported product?

    I have done a similar job that to port a small component from C# to Java manually in 2 weeks time.

    ReplyDelete
  13. We had a project which used asp pages for the client side. We decided to go the j2ee way, and replace it with a spring mvc framework, for various reasons, one of them being to interface with other services within the company which were written in j2ee.

    The approach we took was incremental development. That is we had the old and the new system running in parallel and when we add a new component to replace the old one we switch off that component from the old system.(testing details not included here)

    If new features are a must and had to be added in straightaway and we stil haven't moved that component to the new system, then we add it to the old system.
    However we try to add new features to the new system only as far as it is possible.

    The whole process took us around 7 months for two developers.

    The users weren't too displeased using two systems during this time and they were also reassured by the fact that they still had the old system in the background if the new one failed. i.e. it was easy to back out changes.

    In our case we didn't have to change the core of the underlying database schema by much.

    ReplyDelete
  14. @Thomas Uhl
    We did the Mono POC in March 2009.
    At the time we used Moma (http://www.mono-project.com/MoMA) to understand the C#-Mono gaps we had.
    I asked the architect who did the POC to dig in his archives and see if he can find what were the gaps.
    He has been quite busy in the past 2 weeks, as soon as I'll get the information from him I'll post it.
    From the top of my head i remember issues with WCF and Linq but there were more.

    Note - I don't think the issue is Mono Yes/No. I think that the Mono project is an important project which brings to the open source community .Net technologies. Since i am a big fan of open source i truly appreciate such projects.

    @Anonymous
    The code for accessing the active directory was partially automatically converted.
    For the infrastructure used in C# We built a compatibility library, in which we implemented the missing functionality (like search directory).
    The code for building the search queries and using the classes in the infrastructure was converted automatically.

    Actually we did not bump into any special issues (yet?!)

    @Eric
    In the C# version we expose API for CLI and we have automatic tests running against this API.
    We have the same API exposed in the Java version, so we used the same tests.
    Actually we also have code coverage for these tests but that's for another day....


    Thank you everyone for the inputs and the interest.

    ReplyDelete
  15. Great post! I guess my question is, after the conversion from .NET to Java, what framework does the java code use? Does it automatically convert the code to use Spring & Hibernate framework, for example? On the client side, does it convert C# app to Swing?

    ReplyDelete
  16. @blackrocky
    The converted code does not use any frameworks. It is a pure Java.
    Part of the post processing we did was use EJB and other suitable frameworks we wanted.

    ReplyDelete
  17. Thanks for this post. I believe I spotted it on LWN.net and enjoyed it. Good luck with further development of the Qumaranet management application.

    ReplyDelete
  18. Many people are rumouring that its RHEV-M, you're converting to Java. So, can we assume this true from your post :P?

    ReplyDelete
  19. @Bassu
    yes RHEVM is the project, actually there is an open source project for the next release
    REST API -
    https://fedorahosted.org/rhevm-api/

    Mark McLoughlin and Eoghan Glynn just posted on the first milestone release.

    ReplyDelete
  20. @Bassu
    The API is the only part that is currently open sourced, the implementation will follow.

    ReplyDelete
  21. "Note - I don't think the issue is Mono Yes/No. I think that the Mono project is an important project which brings to the open source community .Net technologies. Since i am a big fan of open source i truly appreciate such projects. "

    Aren't you concerned that Microsoft might set their trolling brigade (Paul Allen and Co) to atack Red Hat as soon as you touch Mono? Or worse, they might just troll your customers once you deploy it, like Microsoft already does with Linux (Novell deal, Amazon, HTC, etc).

    IMHO, those are only reasons why Microsoft is doing Mono. You made a good choice not using it. It's a trap.

    Good luck with RHEV-M

    ReplyDelete
  22. Dude,

    Smart work!! :) ..

    --Hum

    ReplyDelete
  23. switching from C# to java is not an easy task but guys did a wonderful job..i find java complex as compare to C#..right now we are working with .Net as it makes the task easy and reduce lot of coding work which is automically done..but if there is other choice then it must be java..thanks for the useful information of this post..

    ReplyDelete
  24. Hello, i think that this post is very good, i would like to read more about it

    ReplyDelete
  25. I would like read more information about this, is very interesting! Thanks for the information. A worth bookmarking blog. I would be reading your articles regularly from now on.

    ReplyDelete
  26. Nice post. really very interesting things are said in this post. keep up the good job dude.

    ReplyDelete
  27. www.codeproting.com is another online conversion tool which you can use to convert your csharp or .net code to java.

    ReplyDelete
  28. Anyway, I am adding this RSS to my email and could look out for much more of your respective interesting content. Make sure you update this again soon.

    ReplyDelete
  29. This comment has been removed by a blog administrator.

    ReplyDelete
  30. This comment has been removed by a blog administrator.

    ReplyDelete
  31. This comment has been removed by a blog administrator.

    ReplyDelete
  32. Thanks for sharing a such a nice posts.

    Just came across another wonderful application that converts C# source code to C++. CodePorting.Native Cs2Cpp can simplify & automate the source code conversion process, allowing you to publish apps on additional platforms with new every release. I hope it is going to helpful for the rest of community.

    Read More: https://products.codeporting.com/native/cs2cpp

    ReplyDelete