Monday, April 5, 2010

Switching from C# to Java

In 2008 RedHat acquired Qumranet, a startup whose focus was Virtualization. Among other products Qumranet developed a management application for Virtualization.

The management application was written in C# and one of the first tasks we got was to make the management application cross platform, well this was expected considering the fact that the acquisition was done by RedHat...

We started exploring the web looking for ideas how to approach this task. At the beginning things did not look promising most of the references we found for porting projects from one technology to another were about complete failures, the only obvious suggestion that we saw all over was not to change technology and architecture at the same time.

Armed with this important advice we kept digging around and realized there are two different paths we can take. The first was to stick with C#, and the second, surprisingly, was to change technology.

Sticking with C# requires technologies to help us run it on Linux. We found Mono which is an open source implementation of Microsoft's .NET Framework which can be executed on Linux, and the second option we found was Grasshopper which is a project of Mainsoft to compile MSIL to Java Bytecode. The idea is to code in C#, compile it to Java Byte code and run the code on JRE on any JRE enabled platform, Linux included. Cool stuff right?
These 2 solutions were taken off the table, we are on our way to open source our project (Red Hat remember...) and we wanted to use a technology supported by Red Hat. In addition a basic POC we did (POC #1) using MONO proved that the technology was immature at the time and did not meet our needs.

Okay switching technology then. At this point Java looked like the natural choice. It is cross platform, has a lot of enterprise framework ready to use and it's syntax and OO principles are very similar to C# which is an advantage when it comes to training the development team.

Here is a problem for CS guys:
input: 2.5 developers, 4 month period, 100k lines of C# code and a relatively mature management application
output:parity application in java
constraint: during these 4 months 4 developers are adding required features on the C# version
algorithm: ?


After we realized we have no cheat sheet for this problem we considered 3 options:
1. Manual - Writing it all from scratch in java
2. Hybrid - integrate Java modules in the C# application
3. Automatic - automatically convert the code (Yes we believe in miracles, at this point who wouldn't?)

Manual -The obvious option was writing it from scratch, every developer's dream, who does not think that the second time he will write the same code in the most generic flawless way? well nice but no thank you, first of all it is most likely that we will not do the same mistakes again instead we will introduce new bugswhich will probably take forever to fix. In addition we will probably lose whatever maturity the application has. Another con which eventually ruled this option out was that writing from scratch in Java while trying to catch up on the moving target in C# seems impossible to us. We did another POC (POC #2) with a new generic architecture to get an estimate of the time and amount of work for this approach.


Hybrid - The current architecture of the C# version is based on the Command design pattern. Generally we can say that the flows/actions in the system can be mapped to commands. The Hybrid approach was to gradually port the flows from C# to java and during the process have both technologies live side by side. The obvious pros of such an approach are that we can have the system running at all times which can be good for the system maturity, we could also keep developing new features in that period (write them in Java of course) and we could harness the whole team for the conversion (+4 developers to work with us). Looking at the cons this is considered a high risk path, we cannot deliver such a product until the conversion process is complete and the time estimates for this process were kind of random, it actually risked the next release (which originally was planned in C# and was due at the end of the 4 month period), another con was that we realized that for the Hybrid approach we had to have some of the infrastructure ready in Java before we start migrating flows, which turned out to be a lot of code, we might as well port the whole thing. One other difficulty we encountered was that managing database transactions in such a system was very problematic. Anyway the high risk was decisive and yet another POC goes down the drain (POC #2.5, we can't actually call it a POC because it was never executed...)


Automatic - The inspiration for this was an article about Boeing and automatic conversion. Well we thought "if Boeing can do it so can we". Sounds stupid? Well it is.
Luckily for us we did not think that at the time. We looked for automatic conversion tools from C# to java, we actually came across 2 of them one is net2java, an abandoned project, which did not help us much, the second one was Tangible with which we started a POC (POC #3).
At the beginning it looked horrible, we converted the project and got over 50K compilation errors but as I mentioned the Boeing article was inspiring (plus we did not have other options) so we tried looking into the compilation errors, what we found was that some of them were repetitive and by doing a little work we can eliminate some of them.
We started by sending Tangible one or two issues which dominated the errors list, we were pleasantly surprised by Tangible's support, they were so cooperative and fast (or shall I say "he was", Dave, was one very efficient developer) it actually encouraged us to send a second chain of bugs followed by a third and a forth. We ended up filing more than 200 issues (over a 6 month period) of bigger and smaller issues we encountered on our journey to Java.

In addition to the ongoing effort with Tangible we had to dumb down the C# code.
For example we had to remove the usage of the Linq library in the C# side since it is not converted to Java properly.
And in the same time we wrote some "sed" scripts to manipulate the Java output and fix some errors like packaging, adding import statements at the beginning of the class or adding a static data member (logger) in each class.

Obviously there was some manual work as well, but I'll elaborate on the technical details of the conversion process itself in my next post.

It took us around 4 months to stabilize the process and get to a working version of our application in Java. We got to a parity version in Java which passed 90% of our automatic testing. We sometimes still can't believe it.
We did not do a scalability test on the Java version yet, I will blog about it as soon as we have some results.


Although for automobiles the next generation is hybrid for us automatic conversion did the job.

Cheers,
Livnat

36 comments:

  1. Livnat,

    Thanks for sharing this important post.

    We are also currently working on porting a legacy .NET server (Windows + IIS + WCF) to Java EE (Linux + JBoss + EJB3 + JAX-WS) and initially considered Manual and Hybrid approaches.

    We finally choose the Manual approach as we believed that re-designing the architecture is required in our case. I believe that Java has more to offer to enterprise server development than .NET (but I am sure people will argue me with me about it).

    For the Hybrid approach we also considered JNBridge which allows very good .NET <-> Java interoperability and looks very promising.

    We didn't seriously consider the Automatic approach for the same reason you mentioned (will it really work?). WELL DONE!

    One more problematic aspect you didn't mention (and I will love to hear from you) is the political one. Taking a team of (experienced) C# developers and telling them to do Java from now on, to my experience, is not trivial at all!

    Although syntax is similar between the platforms, you have all the different JSRs (EJB, JPA, JDBC, CDI, JAX-WS, JMS, JCA, Servlet, etc.) which are on top of the Java language and are not trivial to master even for experienced Java developers (not to mention new comers to Java).

    Shai

    sl1977@bezeqint.net

    ReplyDelete
  2. My advice to anyone going this way is don't pick the default stack(EJB,JSF,etc). The openness created a lot of diversity in the java platform so there are millions of frameworks and platforms to pick each with up and downsides a job that can be most daunting.

    I'm most confortable with Spring and Grails, but Wicket, Guice, GWT all can be a great fit for your problem and your team.

    ReplyDelete
  3. @Shai
    Thank you for your comment.
    I can relate to the need to re-design and fix flaws in the system, at the same time I think that one of the best decisions we made was not to change technology and design in parallel.
    I think that doing the full switch both technology and design might be too much to handle, maybe having some stepping stones on the way might ease the process a little bit.
    After you switch technology and stabilize your application (sticking to the original design) you have the same application 'only' in another language, that is easier to process mostly for developers who are new to the technology and not less important to the working environment (IDE, build tools like maven, DB etc.).
    Then as a second step you can redesign your system, maybe do it one component at a time. By this time the team is more acquainted with the technology and might feel more comfortable with changing the architecture, same goes for adopting new frameworks, you might want to start with the bare minimum and grow from there.
    Another huge advantage is that by doing one step at the time after the first step you can decide if you want to redesign your system or introduce new features to it, maybe some combination of the two. You might also add stability and maturity to your system in the process.

    About the difficulties of the team, I can say that I am lucky to work with some open minded developers, fact that really made the process more comfortable for us.
    Although to be honest we are still struggling with the change.
    Good luck with your conversion process,
    Livnat

    ReplyDelete
  4. Nice post!
    Good work! :)

    mku.

    ReplyDelete
  5. we ported 260k C++ application to Java with 2 developers in 15 months (still not fully ready for system testing). We changed both language and technology. but additional complexity was that we also changed platform. Initial estimation was 6 months. but it looks all in all it will take 20 months.

    ReplyDelete
  6. Is the current java source tree available for community consumption yet?

    ReplyDelete
  7. Had you considered incremental porting from C# to J#, which is Java syntax, perhaps adding abstractions for to be more Java library compliant (Java collections, etc.), then doing a manual port of the J# to actual Java? This seems like it could have been productive, and yet allow you to keep your system up and running and progressing.

    Perhaps rearchitecting to Spring.NET (I don't know how compatible Java Spring and Spring.NET, but I would imagine they are quite similar).

    ReplyDelete
  8. Great post :)
    Keep us updated.

    ReplyDelete
  9. @Anonymous
    The code is not available for community consumption yet, but it is on our road map.

    @Will Hartung
    We did not consider using J# or Spring.Net as an intermediate state, interesting idea though.
    Two things that come to mind -
    1. We don't have in house knowledge of these technologies.
    2. Having another technology as a stepping stone might increase risk.

    ReplyDelete
  10. JNBridge looks like stuff that allows to connect C# to Java, which is not what was asked for here.

    ReplyDelete
  11. Why was Mono immature? We are using it heavily within our projects.

    ReplyDelete
  12. I imagine that some of your .NET code was relying on .NET libraries to talk to Active Directory domains. Did this code get automatically converted or did you have to manually convert it?
    Also did you bump into any issue with regards to Java Active Directory libraries?

    ReplyDelete
  13. Can I please ask how was testing being done on the ported product?

    I have done a similar job that to port a small component from C# to Java manually in 2 weeks time.

    ReplyDelete
  14. We had a project which used asp pages for the client side. We decided to go the j2ee way, and replace it with a spring mvc framework, for various reasons, one of them being to interface with other services within the company which were written in j2ee.

    The approach we took was incremental development. That is we had the old and the new system running in parallel and when we add a new component to replace the old one we switch off that component from the old system.(testing details not included here)

    If new features are a must and had to be added in straightaway and we stil haven't moved that component to the new system, then we add it to the old system.
    However we try to add new features to the new system only as far as it is possible.

    The whole process took us around 7 months for two developers.

    The users weren't too displeased using two systems during this time and they were also reassured by the fact that they still had the old system in the background if the new one failed. i.e. it was easy to back out changes.

    In our case we didn't have to change the core of the underlying database schema by much.

    ReplyDelete
  15. @Thomas Uhl
    We did the Mono POC in March 2009.
    At the time we used Moma (http://www.mono-project.com/MoMA) to understand the C#-Mono gaps we had.
    I asked the architect who did the POC to dig in his archives and see if he can find what were the gaps.
    He has been quite busy in the past 2 weeks, as soon as I'll get the information from him I'll post it.
    From the top of my head i remember issues with WCF and Linq but there were more.

    Note - I don't think the issue is Mono Yes/No. I think that the Mono project is an important project which brings to the open source community .Net technologies. Since i am a big fan of open source i truly appreciate such projects.

    @Anonymous
    The code for accessing the active directory was partially automatically converted.
    For the infrastructure used in C# We built a compatibility library, in which we implemented the missing functionality (like search directory).
    The code for building the search queries and using the classes in the infrastructure was converted automatically.

    Actually we did not bump into any special issues (yet?!)

    @Eric
    In the C# version we expose API for CLI and we have automatic tests running against this API.
    We have the same API exposed in the Java version, so we used the same tests.
    Actually we also have code coverage for these tests but that's for another day....


    Thank you everyone for the inputs and the interest.

    ReplyDelete
  16. Great post! I guess my question is, after the conversion from .NET to Java, what framework does the java code use? Does it automatically convert the code to use Spring & Hibernate framework, for example? On the client side, does it convert C# app to Swing?

    ReplyDelete
  17. @blackrocky
    The converted code does not use any frameworks. It is a pure Java.
    Part of the post processing we did was use EJB and other suitable frameworks we wanted.

    ReplyDelete
  18. Thanks for this post. I believe I spotted it on LWN.net and enjoyed it. Good luck with further development of the Qumaranet management application.

    ReplyDelete
  19. Many people are rumouring that its RHEV-M, you're converting to Java. So, can we assume this true from your post :P?

    ReplyDelete
  20. @Bassu
    yes RHEVM is the project, actually there is an open source project for the next release
    REST API -
    https://fedorahosted.org/rhevm-api/

    Mark McLoughlin and Eoghan Glynn just posted on the first milestone release.

    ReplyDelete
  21. @Bassu
    The API is the only part that is currently open sourced, the implementation will follow.

    ReplyDelete
  22. "Note - I don't think the issue is Mono Yes/No. I think that the Mono project is an important project which brings to the open source community .Net technologies. Since i am a big fan of open source i truly appreciate such projects. "

    Aren't you concerned that Microsoft might set their trolling brigade (Paul Allen and Co) to atack Red Hat as soon as you touch Mono? Or worse, they might just troll your customers once you deploy it, like Microsoft already does with Linux (Novell deal, Amazon, HTC, etc).

    IMHO, those are only reasons why Microsoft is doing Mono. You made a good choice not using it. It's a trap.

    Good luck with RHEV-M

    ReplyDelete
  23. Dude,

    Smart work!! :) ..

    --Hum

    ReplyDelete
  24. switching from C# to java is not an easy task but guys did a wonderful job..i find java complex as compare to C#..right now we are working with .Net as it makes the task easy and reduce lot of coding work which is automically done..but if there is other choice then it must be java..thanks for the useful information of this post..

    ReplyDelete
  25. Hello, i think that this post is very good, i would like to read more about it

    ReplyDelete
  26. I would like read more information about this, is very interesting! Thanks for the information. A worth bookmarking blog. I would be reading your articles regularly from now on.

    ReplyDelete
  27. Nice post. really very interesting things are said in this post. keep up the good job dude.

    ReplyDelete
  28. www.codeproting.com is another online conversion tool which you can use to convert your csharp or .net code to java.

    ReplyDelete
  29. What I think that it is wonderful idea of switching from C# to Java. Certainly, Java language is easier than C#. I did work on both language and always feel more comfortable to work with Java language.
    Cloud experts

    ReplyDelete
  30. Anyway, I am adding this RSS to my email and could look out for much more of your respective interesting content. Make sure you update this again soon.

    ReplyDelete
  31. This comment has been removed by a blog administrator.

    ReplyDelete
  32. Otr214428
    INTERNATIONAL CONCEPT OF WORK FROM HOME
    Work from home theory is fast gaining popularity because of the freedom and flexibility that comes with it. Since one is not bound by fixed working hours, they can schedule their work at the time when they feel most productive and convenient to them. Women & Men benefit a lot from this concept of work since they can balance their home and work perfectly. People mostly find that in this situation, their productivity is higher and stress levels lower. Those who like isolation and a tranquil work environment also tend to prefer this way of working. Today, with the kind of communication networks available, millions of people worldwide are considering this option.

    Women & Men who want to be independent but cannot afford to leave their responsibilities at home aside will benefit a lot from this concept of work. It makes it easier to maintain a healthy balance between home and work. The family doesn't get neglected and you can get your work done too. You can thus effectively juggle home responsibilities with your career. Working from home is definitely a viable option but it also needs a lot of hard work and discipline. You have to make a time schedule for yourself and stick to it. There will be a time frame of course for any job you take up and you have to fulfill that project within that time frame.

    There are many things that can be done working from home. A few of them is listed below that will give you a general idea about the benefits of this concept.

    Baby-sitting
    This is the most common and highly preferred job that Women & Men like doing. Since in today's competitive world both the parents have to work they need a secure place to leave behind their children who will take care of them and parents can also relax without being worried all the time. In this job you don't require any degree or qualifications. You only have to know how to take care of children. Parents are happy to pay handsome salary and you can also earn a lot without putting too much of an effort.

    Nursery
    For those who have a garden or an open space at your disposal and are also interested in gardening can go for this method of earning money. If given proper time and efforts nursery business can flourish very well and you will earn handsomely. But just as all jobs establishing it will be a bit difficult but the end results are outstanding.

    Freelance
    Freelance can be in different wings. Either you can be a freelance reporter or a freelance photographer. You can also do designing or be in the advertising field doing project on your own. Being independent and working independently will depend on your field of work and the availability of its worth in the market. If you like doing jewellery designing you can do that at home totally independently. You can also work on freelancing as a marketing executive working from home. Wanna know more, email us on workfromhome.otr214428@gmail.com and we will send you information on how you can actually work as a marketing freelancer.


    Internet related work
    This is a very vast field and here sky is the limit. All you need is a computer and Internet facility. Whatever field you are into work at home is perfect match in the software field. You can match your time according to your convenience and complete whatever projects you get. To learn more about how to work from home, contact us today on workfromhome.otr214428@gmail.com and our team will get you started on some excellent work from home projects.


    Diet food
    Since now a days Women & Men are more conscious of the food that they eat hence they prefer to have homemade low cal food and if you can start supplying low cal food to various offices then it will be a very good source of income and not too much of efforts. You can hire a few ladies who will help you out and this can be a good business.

    Thus think over this concept and go ahead.

    ReplyDelete