Software Development
Staying On Top Of The Game – Podcasts for Developers
Jul 17th
As a professional software developer, or for anyone working in IT for that matter, it’s hard to stay on top of the game: Staying competitive includes following up on the latest industry trends and knowing about the latest technologies. But you better still be productive in your job and all the while maintain a certain social life – chances are you won’t get a lot of sleep.
Reading books, blogs, etc is of course a great way of going about it, but then again: Who really has time to read nowadays. I am trying to get through “The Art of Unit Testing” for almost a month now. It’s by no means a big book (approx 250 pages of content) – I just don’t find the opportunity to read more than 5 pages in one sitting.
For me, podcasts present a great alternative. There are a number of very high quality, development and technology related podcasts out there with an incredible information density. I listen to the following on my IPod whenever I am “in between” things (on the way to work, while running, shopping, etc) or while performing more or less mindless tasks (cleaning, cooking, …):
Cranky Geeks: In a very cut and dry style that is right up along my alley, John C. Dvorak and guests discusses latest events in the industry. Every podcast packed with information and gives you a good idea of “what’s happening”: Industry rumors, new technologies, new devices, you name it. John is not talking about software development specifically, so it’s a great complement to the podcasts mentioned below. The cast is very easy and entertaining also to listen to and with a length of half an hour ideal for the daily commute.
Java Posse: You can tell that Tor Norbye (Sun Microsystems), Carl Quinn (Netflix), Dick Wall (Navigenics) and Joe Nuxoll (independent) are having fun producing this podcast. If you want to know what’s going on in the Java community, there is no better place to go than “Java Posse”. It’s an hour long, but well structured and the chemistry between the hosts makes it easy and entertaining to listen to.
Hanselminutes: Focussing mainly on the .Net and Windows world, Scott Hanselman discusses utilities, tools and gives practical how-to advice. The podcasts are a nice mix of .Net/Windows specific subjects and more generic topics such as agile development and project management. The guests are high profile and Scott is a great host who manages to distill and compress valuable in-depth information within a 30min time box.
Software Engineering Radio: This one is a slightly different beast. While all the previously mentioned postcasts focus on being right on top of what’s happening, the SE Radio hosts and their thick accents go back to basics: Their guests talk about SQL and release management, the books they have written and “old news” such as MDA, etc.
The episodes are long (1h) and the hosts are by no means skillful interviewers: You can easily zone in and out of the podcast and answer a phone call without having to rewind. The artificial laugh’s at not-funny jokes get annoying pretty quick and sometimes there is just no clear direction in the interview.
However, I am a firm believer in knowing the roots and basics of what we do in our industry, and for that reason alone I follow this podcast. I would classify it as “interesting background noise” and I do take home little bits and pieces from every episode.
StackOverflow: Joel Spolsky, a software developer in New York City who is probably best known in the community for his blog Joel on Software, and his collegue Jeff produce this podcast which talks about…well… mostly themselves. The podcast doesn’t necessarily convey a lot of new new information and I treat this podcast as a replacement for mp3s or radio. I personally like to to listen to others talk about their pet project and the associated worries, problems and victories.
Cutting Corners Doesn’t Buy You Anything
Jan 30th
There has been a lot of talk of identity theft, security threats and securing private data lately. You’ve all seen reports of a buggy company websites which exposes private data to the public, phishing scams or an employee using private data for personal use. It makes you think about all of the personal data that you make available each day, whether that be to social media sites, for online purchases or even your phone and utiliy provider. How secure is it?
From a service provider perspective, the minute we store a user’s private data, for example their username and password, we’ve taken on the responsibility of securing that data too. Let’s say a hacker somehow obtains a list of all our usernames and passwords. Either it was an inside job by someone who had access to the database, or the database was accidentally exposed to the public web. It doesn’t matter how. It just happened.
Now as a customer I wouldn’t be happy about my prvate data becoming available. However, I would expect the owner of the application in question to have taken sufficient measures to render it useless to third parties. At the very least, the password is going to be encrypted, right?
RIGHT?????
The answer – from experience – is way too often “unfortunately not”!
Don’t get me wrong, most system designers, developers and project managers are smart people. They think long and hard and often times come up with a reason to avoid taking the standard security measures which everyone sort of knows about.
They are wrong!
How can I be so sure? Because, when it comes to web-app security, cutting corners doesn’t buy you anything. It doesn’t save you coding time (in the long run). It doesn’t give your users a better experience. All it does is weaken the security of your web site, needlessly putting your users, your employer, and yourself at risk.
As the developer, you might think that it’s relatively unimportant if your user’s application password is exposed as plain text. After all, what’s an attacker going to do with, for example, forum credentials? Post angry messages on the user’s behalf? And what about the small business that requires private data from their customers to sign up for a company newsletter? Same thing here. The consequences of exposed data seem small.
But most users tend to re-use the same passwords, probably because they can’t remember the two dozen unique usernames and passwords they’re forced to have. So if you obtain their forum password, it’s likely you also have the password to something a lot more dangerous: online banking, PayPal, etc
So please allow me to state the obvious:
If you are storing plaintext passwords in a database, you are making a mistake!!!
A lot of developers – often times too late – move to hashed passwords. Even this is not enough to thwart a determined attacker. Hashes alone are better than plain text, but barely. Hashing the passwords prevents plaintext exposure, but it also means you’ll be vulnerable to astonishingly effective rainbow table attacks.
The solution here is salted hashes based on computationally expensive(!) hash functions which were designed for password applications. If you do not know what a “salted hash” is or why you should waste precious compute cycles for password storage – you need some help. Hire someone that knows. Otherwise, it’s only a matter of time before you pay for this…
So please let me take this opportunity to ask if you know of (or perhaps work on) any software systems that store passwords plain text in a database. If so, fix your software now:
- Salt and hash each and every password (use a computationally expensive hashing function such as bcrypt that was designed for password applications)
- Store the salt and hash – not the password – in your database.
- Throw the password itself away.
- Educate your fellow developers! This might feel like “development 101″,but not everyone knows what you know. Really!
You will be glad you did…
The Power of Facelets (Part 1 – Motivation)
Dec 15th
Ever since Java Server Faces (JSF) were released, we utilized them successfully for many web application projects. While being conceptually really nice and well thought through, one problem with JSF has always been the mismatch with JSP (Java Server Pages).
The issue is how to integrate JSP’s dynamic content into JSF’s component-based model. JSPs are singularly focused on generating dynamic output, whereas JSF requires JSP to coordinate building a component model. The disjunct occurs because that task is beyond the original intention of JSP.
Even though most JSF developers simply learn to navigate such problems, it made the usage of JSF in large projects a huge pain and always resulted in ugly code that was difficult to read and maintain. Far from what you want to achieve with a new technology.
The Problem with JSPs
…Well – maybe not JSP per se. JSPs are great for their intended use: Mixing both static and dynamic content made available by other parts of the application – servlets, plain java code, databases, etc.
But keep in mind that – historically – JSPs are nothing more than “fancy servlets” with some added semantic sugar. Their sole purpose of existence is to generate a response to a request; a JSP page is processed in one pass from top to bottom, with JSP action elements processed – here it comes – in the order in which they appear in the page.
JSF are component based and have a much more complex life cycle. They are created, asked to process their input (if any), and then asked to render themselves (I am simplifying here). For JSF to work well, these three things must happen separately in a well-defined order, but when JSF is used with JSP, they don’t. Instead, the component creation and rendering happens in parallel, causing all kinds of problems.
The fact that both JSP and JSF components add content to the response is another cause for a lot hard to track down issues. Unless you understand the difference between how these two technologies write to the response, you’re in for a lot of surprises – such as content appearing out of order or not at all.
There are severe limitations on how JSF and non-JSF tag libraries can be mixed in a JSP page. On top of that, developing a JSF application requires understanding the event-driven component model.
When learning JSF, a JSP developer now has to not only understand those (likely unfamiliar) concepts, but also learn the pitfalls of how to use the two technologies together and – more importantly – how not to. It’s a lot to learn and often unintuitive.
Lets have a look at a very simple example that illustrates the issue with parallel component creation and rendering. Say you want to create a label for a checkbox – bread and butter which we developers don’t even want to think about, right? Intuitively you would write something like this:
There is no intuitive reason this shouldn’t work, but if run this example, you’ll notice that no <label> element is rendered the first time you request the page, but if you request the page a second time, a <label> element is rendered.
This strange behavior has to do with the combination of JSF and JSP. When JSF receives the first request for the page, the component tree for the view represented by the page doesn’t yet exist. JSF creates a tree with just a UIViewRoot at the top and forwards the request to the JSP page to add the real components.
The JSP container now processes the page and invokes the JSF action tag handlers (here it comes again:) in the order they are encountered. A JSF tag handler looks for the JSF component it represents in the component tree. If it can’t find the component, it creates it and adds it to the component tree. It then asks the component to render itself. This means that on the first request, the components are created and rendered in parallel. On subsequent requests, the component tree exists, so no new components are created; the tag handlers just ask the existing components to render themselves.
On the first request, the >h:outputLabel< action creates its component and asks it to render itself. To do so, it needs to find the component identified by the for attribute, but this component doesn’t exist because the action element that creates it appears after the <h:outputLabel> element and hasn’t been invoked yet. Hence, the component created by <h:outputLabel> action can’t render its <label> element. On the second request, all components exist, so the component represented by the <h:outputLabel> finds the component the label belongs to and happily renders the label element.
Of course in this case there are easy workarounds, but hey: Why should I “work around’ something as simple as adding a label to a checkbox? Furthermore – as you start going beyond “Hello World” pages, you encounter more and more examples like the one above, and many of them aren’t as easy to understand. Matters get worse once you start using AJAX, A4JSF and similar technologies.
In a recent project, the code for the view pages were – out of necessity – plastered with <f:verbatim>pages – creating pages with unbalanced tags (hey – we had no choice) and making it almost impossible to work with.
The solution was – of course – to dump JSP altogether and move to Facelets – more about this in the next article.
AJAXified Location Search Text Field in 15min Using MS Life Search, Java and EXTJS
Dec 15th
A recent project involved a Microsoft Virtual Earth mashup. As it turns out, the initial component set shipped with the Virtual Earth (VE) SDK is rather limited and – quite frankly – very ugly.
One of the things needed was an input field the user can utilize to type in an address or location, which gets then of course shown in the map. Ideally this input field would provide a “suggestion drop down” containing places known to VE. As it turns out, this is accomplished using…
- Java Servlets for the middle tier (project requirement)
- EXTJS for the components and the AJAX communication
- The MS Virtual Earth Life Search web service
The Servlet
Wrapping the call to the VE web service into a servlet has several advantages: We can perform caching of the requests, thus significantly lowering the response time of the service while being able to create nice statistics of what people are searching for. It also allows us to process both the search string and the results on the server to perform e.g. filtering, limiting searches to geographic areas, etc.
Calling the web service to perform the address lookup is very easy:
private static String URL_PARAMETER_PREFIX ;
private static String URL_PARAMETER_POSTFIX;
static{
try {
URL_PARAMETER_PREFIX = URLEncoder.encode("a", "UTF-8") + "=" + "&" + URLEncoder.encode("b", "UTF-8") + "=";
URL_PARAMETER_POSTFIX = "&" + URLEncoder.encode("c", "UTF-8") + "=" + URLEncoder.encode("0.0", "UTF-8");
URL_PARAMETER_POSTFIX += "&" + URLEncoder.encode("d", "UTF-8") + "=" + URLEncoder.encode("0.0", "UTF-8");
URL_PARAMETER_POSTFIX += "&" + URLEncoder.encode("e", "UTF-8") + "=" + URLEncoder.encode("0.0", "UTF-8");
URL_PARAMETER_POSTFIX += "&" + URLEncoder.encode("f", "UTF-8") + "=" + URLEncoder.encode("0.0", "UTF-8");
URL_PARAMETER_POSTFIX += "&" + URLEncoder.encode("g", "UTF-8") + "=" ;
URL_PARAMETER_POSTFIX += "&" + URLEncoder.encode("i", "UTF-8") + "=" ;
URL_PARAMETER_POSTFIX += "&" + URLEncoder.encode("r", "UTF-8") + "=" + URLEncoder.encode("0", "UTF-8");
} catch (UnsupportedEncodingException e) {
URL_PARAMETER_POSTFIX = null;
URL_PARAMETER_PREFIX = null;
e.printStackTrace();
}
}
private String performMSLiveSearch(String address){
if (liveSearchURL == null
|| URL_PARAMETER_PREFIX == null || URL_PARAMETER_POSTFIX == null ){
// log error here
return "";
}
StringBuilder requestResult = new StringBuilder();
try {
String paramteterString = URL_PARAMETER_PREFIX
+ URLEncoder.encode(address, "UTF-8")
+ URL_PARAMETER_POSTFIX;
// Send request
URLConnection conn = liveSearchURL.openConnection();
conn.setDoOutput(true);
OutputStreamWriter wr = new OutputStreamWriter(conn.getOutputStream());
wr.write(paramteterString);
wr.flush();
// Get the response
BufferedReader rd = new BufferedReader(new InputStreamReader(conn.getInputStream()));
String line;
while ((line = rd.readLine()) != null)
requestResult.append(line);
wr.close();
rd.close();
} catch (Exception e) {
// log exception here
e.printStackTrace();
}
System.out.println(requestResult.toString());
return requestResult.toString();
}
Instead using heavyweight XML parsers, we can extract the needed information fast and simple using regular expressions:
private Pattern ambiguityListPattern = Pattern.compile(".*?new Array\\('(.*?)',.*?\\)");
private Pattern geoLocationPattern = Pattern.compile(".*?AddGeoLocation\\('(.*?)',.*?\\)");
private String[] analyzeSearchResults(String searchResultString){
if (searchResultString == null)
throw new IllegalArgumentException("Parameter must not be null!");
// unfortunately we are stuck with 1.4, so no generics
ArrayList results = new ArrayList();
// extract the "ambiguity list"
Matcher m = ambiguityListPattern.matcher(searchResultString);
while (m.find())
results.add(m.group(1));
// extract the "geolocation list"
m = geoLocationPattern.matcher(searchResultString);
while (m.find())
results.add(m.group(1));
return (String[])results.toArray(new String[results.size()]);
}
To finalize the servlet, we implement the doPOST method as follows:
protected void doPost(HttpServletRequest req, HttpServletResponse resp) throws ServletException, IOException {
String searchAddress = req.getParameter("address");
PrintWriter out = resp.getWriter();
// chaching and search string processing goes here
String[] results = analyzeSearchResults(performMSLiveSearch(searchAddress));
LocationSearchCache.getInstance().addSearchResults(searchAddress, results);
// format output for EXTJS array reader
out.print("[");
for (int i=0; i< results.length; i++){
if (i>0)
out.print(",");
out.print("['");
out.print(results[i]);
out.print("']");
}
out.println("]");
out.close();
}
This leaves us with the HTML and Javascript, which – thanks to the great EXTJS library, is a breeze:
Ajax Virtual Earth Search Test
That’s it! All the magic is done in…
Ext.onReady(function() {
var store = new Ext.data.Store({
proxy: new Ext.data.HttpProxy({
url:SERVLET_ADDRESS, method: 'POST'}),
reader: new Ext.data.ArrayReader({}, [{name: 'address'}])
});
var combo = new Ext.form.ComboBox({
store: store,
queryParam:'address',
displayField:'address',
hideTrigger:true,
typeAhead: false,
triggerAction: 'all',
emptyText:'City, Address or Postal Code',
selectOnFocus:true
});
combo.applyTo('addressCombo');
});
That’s it! We end up with a very flexible component that can be easily plugged into any page, or even modularized into a JSF component.
We used the doGet method of the servlet to spit out statistics about the current state of the query cache, which also only takes a few lines of code.
Security Exploit for for Ajax Based Applications: JavaScript Hijacking
Dec 15th
In their recent paper (download here) Brian Chess, Yekaterina Tsipenyuk O’Neil, Jacob West from Fortify Software describe a new type of eavesdropping attack against Ajax-style Web applications called Javascript Hijacking.
The attack is possible because Web browsers don’t protect JavaScript the same way they protect HTML – if a Web application transfers confidential data using messages written in JavaScript, in some cases the messages can be read by an attacker.
The authors analyzed 12 popular Ajax frameworks and show that most of them do nothing to prevent JavaScript hijacking. Some actually require(!) a developer to create a vulnerable server in order to function.
Like so many of these sorts of vulnerabilities, preventing the class of attacks is easy. In many cases, just a few additional lines of code are required. And like so many software security problems, programmers need to understand the security implications of their work so they can mitigate the risks they face. But it seems that is that JavaScript hijacking won’t be solved so easily, because – especially in the web application world – developers often times don’t understand the security implications of their work and thus nothing in their code will prevent the attacks.
J2EE – Java Server Faces (JSF) and EJB3 – for Legacy Modernization
Dec 15th
We just finished a project with the objective to rewrite and modernize a legacy web application using JSF and EJB3. The new application was to provide a rich, interactive map component as well as a “desktop-like” user experience with interactive tables, split as well as tabbed panes, etc. Let’s have a quick look at the two main tools used for this project:
Enterprise Java Beans (EJB) 3
It’s always a headache if you don’t have control over your data model. Especially when the model is bad. Especially when the database model is really bad – carried over from the old mainframe days and grown (note the avoidance of the word “matured”) ever since. EJB3 handled this job extremely well. Only 4% of project time was spent on backend implementation, a fair share of that on “Mock Data Services” for testing purposes. Nice…
Java Server Faces
Introduced as an event-driven component model for web application development, Java Server Faces are similar in spirit and function to the models used for standalone GUI applications (like Swing).
JSF shows the signs of a typical “Sun-Version-1″: Nice, well thought through concepts, combined with a very basic, almost rudimentary component set, forcing you to either use third party libraries (often times impossible in large corporate environments due to policies, etc) or “reinvent the wheel” and build your own component sets.
I like working with JSF for many reasons, but once things move beyond “Hello World” and “Guess-The-Number”, you have to know the internals of JSF very well or you will spend many hours debugging issues that are far from being intuitive.
Don’t get me wrong, I am enthusiastic about JSF and think it’s one of the better frameworks out there for large scale web applications, yet I am very much looking forward to version 2….
Some of the things I like about JSF:
The concept of “Component Renderer”, common to desktop frameworks, allows for easy and elegant combination of JSF with existing WEB 2.0 technologies like AJAX.
Being able to reuse existing component renderer for custom components makes it very easy to create new, complex and reusable components for complex and rich GUI elements.
Security: JSF fits very nicely with security libraries like JAAS. Used properly (custom component renderer and phaselistener), the front end developer can be shielded from any security related code without compromising security. I still love the little things like our “securityOutputLink”, which renders a simple text string if the user does NOT have access to the link target, and a regular link otherwise. In the backend, the decision is based on the same xml file which links user roles to page access, all with a handful of code lines. Sweet!
Configuration of navigation rules and backing bean properties in faces-config.xml is a very nice concept and allows “out of the box” for different application configurations for development, testing and production, which is a real timesaver.
On the Minus Side
The very bad integration of JSF with JSP and JSTL. This is certainly my number one criticism when it comes to JSF. As someone put it so nicely: Trying to combine JSF and JSP is like trying to shoehorn a foot into a glove: it’s possible, but it’s really just a stopgap measure until something better comes along. It would be nice, real nice to have a powerful expression language at your disposal. Sometimes you just want a loop without having to write your own component renderer, or conditional execution on a page without having to worry and work around the different variable scopes and execution times of JSF and JSP. Read here for more details
Small and very basic component set. Sortable, pagable tables with row highlighting and selectable rows and columns are certainly bread and butter for every real web application. Since the number one reason for using frameworks like JSF is to make a developer’s life easier, it’s hard to explain why I have to still write this myself (or switch implementations).