I mentioned a few days ago that we're looking for a J2ME person, well we're now also looking for one or more Senior J2EE peeps, as well as a Database/Unix person. As before, we need solid, technically well-rounded people with the right disposition to cope in a small, startup-style environment. We've got solid funding from big-name VC's and we have strong relationships with the mobile network companies, so this isn't a fly-by-night outfit. But we are very much a young company in an emerging field.

The biggest problem I'm seeing in recruiting for us is that many people don't have the stomach to work in this kind of environment. That's fine, not everybody is suited to fly fighter jets, the world needs airline pilots to handle the routine stuff. For me, I can't stand working in big companies where I'm not allowed to deal with an entire system, and I know there are others like me.

Several years ago I had a play with Orion, which seemed like the up and coming Java app server, the preferred tool for the True Techie. It was well-built, running very fast and incorporating all the latest standards, and on top of all that it was cheap, $1,500 per server (regardless of how many CPUs), and freely downloadable and usable for development use with no registration keys, expiration, or other cripplement.

But then Oracle licensed Orion and used it to build their App Server product. While on the face of it this shouldn't have affected IronFlare's delivery of Orion, other than to give them some cash and "enterprise" credibility. But apparently the licensing required IronFlare's developers to spend a fair amount of time supporting Oracle in some ways or other, so their own product has slipped.

Basically, Orion stagnated for a few years, and fell of my radar as well as that of the Java community in general.

But over the past year or so they seem to geared back up, and have the potential to come back into the lead of the pack, especially among more technically driven projects. It makes a lot of sense for our company. We use Weblogic, but WL has a number of disadvantages, particularly that it's massively expensive, it's bulky and slow, and it's tied to commercial distributions of Linux, namely Red Hat.

Orion, meanwhile, is still cheap, with a developer-friendly licensing model. It's still fast, loading up within a second or two on my laptop, and quite simple. It does have its drawbacks; the documentation is only so-so, and there's no real admin interface.

Configuration is old-skool, XML files with no Web/GUI configuration tool, which might put some people off but to me is perfectly fine - text files are much easier to manage, especially for complex, multi-server systems, than having to remember a list of places to point and click.

Logging is pretty minimal, with few options that I've found so far to turn up debugging levels, so figuring out why my application has trouble loading isn't easy. I also haven't found a remote console to track performance indicators. There is a rudimentary GUI console that can be started along with the server itself, but it's local to the server machine, and closing it shuts the server down; so while it's useful for local development, it's useless on a production server. Orion may have JMX/SMTP hooks, if so that would at least give a way to build something to the purpose. But if not this is probably the point that would make it hardest to sell Orion into my company as a replacement for Weblogic.

Another disadvantage is that Orion has yet to catch up to the standards curve, it's still based on J2EE 1.3, including Servlets 2.3 and JSP 1.2.

There's an opportunity for Orion to steal a march on Weblogic if they get their shit together quickly. There are tons of companies around still using Weblogic 6.1, pondering the upgrade to 8.1. That's an expensive upgrade, both in licensing costs and in having to port to the newer APIs. Orion offers a way to cut licensing by at least 90%, and the Weblogic API changes make it no more of a pain than upgrading to 8.1 would. They just need to polish around the edges and they could make a serious leap.

I assume so anyway. For a couple years I've been trying to persuade family visiting from the States that I really did see a restaurant called "Phat Phuc", but I could never remember where it was. The other day when my wife dragged me out to Chelsea I finally found it, or at least the sign.

We're mostly a server-side tech shop - we run a set of applications on Weblogic which users interact with using SMS and WAP, and which makes calls out on the back end to standard web sites. So essentially we provide a bridge so services normally available on the web can be easily provided to mobile phone users. We don't currently have any phone-based applications, but we're looking to do so now.

So if you're an experienced J2ME programmer and can work in London, give me a shout. Keep in mind the "experienced" part - we've got people who have done a bit of J2ME programming, what we need is someone who is initimately familiar with writing and debugging apps that run across many phones. They also need experience with animation, and will need to integrate with server-side applications, so the more you know about HTTP, socket programming, and the like, the better. In general, our company expects programmers to have a solid grounding in CS fundamentals (our founder is a former CS professor).

Oh yeah, we're also looking for server-side Java peeps as well. You don't need experience in the mobile area, although it wouldn't hurt. Mainly you should be into J2EE (we use servlets, Session beans, and JMS mainly, no JSP and no entity beans), and be solid enough to contribute to different parts of the system.

I've been working at a new job for a little while now, and it's a refreshing change from my old job. As I've said before, I quite like my old company, but I got stuck in a rut. I've worked both sides of the programmer/sysadmin line for as long as I've had technical jobs, but the danger of being a sysadmin is that it's too easy to get mired in "IT support", especially at a small company. Herding application servers was fine, but after a reorg I found myself working for a boss who wanted my main focus to be things like new versions of Active Directory, Exchange, and backup systems for laptops - and this is not the direction I want for my life.

So I was really hankering to get back into programming, and I obviously switched just in the nick, considering how hard it was to get through the screens of ignorant recruiters. It didn't matter what I was doing 2 years ago, when they didn't see the right acronyms on my current job they balked at sending my CV through. As far as they were concerned I was a systems administrator, sure I can install and run EJB servers, but my CV doesn't say I have been writing EJB's, so I'm not a programmer. They typically held my CV back for weeks until "better" candidates didn't make the cut, sending me in when they didn't have anything else. They were quite surprised to find that I was actually able to land offers for programming roles. People on the sharp end realize that someone who has done both programming and systems management can be quite useful, but it's not the kind of thing that typically ends up in a job description.

The irony is that now I'm involved in interviewing the kinds of people that recruiters do think are suitable. There is a glut of people who have J2EE on their CV, but for whom this means they have spent the past 3 years working in a large company working on just one thing. I've actually seen people whose entire job was to write stateless session beans. They haven't done much database work (someone else's job), they don't know anything about threading (someone else did that), and they can't write an ant build script (that was a different department), but they know J2EE.

Anyway, I'm enjoying my new role. I'm head of the development team, which is a jolt from heading a small sysadmin team. All of the guys on the team are solid, hard core developers, and given that the system they've built over the past couple years is very involved and not documented for an outsider to learn, I've got some catching up to do. On the other hand, my job here isn't so much to be the alpha geek programmer as it is to be the coordinator between product management and developers. So I'm getting the development process under control and getting a handle on priorities and such.

What's especially cool is that I'm learning a new area of technology. Where most of my work in the past has been web and email focused, now I'm involved in WAP and SMS. Although where the rubber hits the road there's not much difference - it's the same servlets with a different view - there are plenty of new issues involved. At the moment I need to get a handle on various issues with different combinations of phones and mobile operators.

For instance, it turns out Orange had been selling Nokia 6600 phones without Thawte SSL certificates, which means people with those phones can't use our service. This is pretty fucked up considering Thawte are a major provider of SSL certificates. Orange blames it on Nokia, but in fact they install their own version of the OS, and Google doesn't turn up anybody else complaining about 6600 phones not being able to use their secure sites, so you do the math.

One of my company's mail servers got blacklisted by SpamCop last week, which led to our learning a new mail server configuration tweak that's needed in today's spam infested world. We do "e-marketing" mailings, but these are mostly sent to people who specifically sign up for them on corporate web sites - our cients tend to the big and bad-publicity/legal action averse, so everything tends to be very strictly above board. But it turned out the servers we use for this weren't blacklisted, it was the one we use for regular user emails to/from our staff.

We got blacklisted for 48 hours based on a "spamtrap"; an email coming from one of our servers was sent to an email account set up by SpamCop strictly to detect unsolicited mail. The idea is that the spamtrap email address is never given out in a context that would attract legitimate, solicited email. Unfortunately SpamCop is very tight with details on their processes for spamtrap addresses, and there is a lot of variation in how such addresses can be managed.

The strictest spamtrap policy would be simply to set up an address on a server, but never publish it in any way. However some spamtrappers might carefully leak the address in certain controlled ways, for example putting it on an HTML page somewhere for address harvesting crawlers to find, or maybe even using it to send an email to a known address-list seller knowing they'll add it to their lists.

A potential problem with all of these is that spammers and email worms tend to forge email addresses and the sent-from lines that track an email's progress through the Net, and these days they use legitimate addresses and servers. So if they pick your address or your email server and use it in an email to a spamtrap, you can potentially end up on a blacklist.

As it turns out, SpamCop seems to avoid falling for this. The trick that got us blacklisted was actually more subtle, and is a side effect of collateral spam. Collatoral spam is the flood of bounces, "our system detected a virus in your message" alerts, and other useless crap that gets sent to an address that is used as a patsy by the spammers and email worms. If a spamtrap address is used as a patsy address in a spam or worm that is sent to your system, if your system automatically sends a bounce or "you sent us a virus" message, you'll be blacklisted.

At first glance, this is a shitty thing to do in the case of bounces. Mail servers that automatically send "you have a virus" emails to addresses that have been forged in incoming emails deserve to be slammed, they serve no purpose other than to help these people spread chaos to wider audience. My own mail server has filters that refuse to accept delivery of messages that look like a virus alert, with a message explaining why this is stupid.

But bounces are a necessary part of email transportation, in fact they're required by email standards, so at first SpamCop's policy looks like it punishes you for complying with the standards. As it turns out though, there are two ways to bounce mails.

The old way is to take delivery of an email and close the SMTP connection before checking whether the email is addressed to a legitimate user. If not, a new email is created and sent to the appropriate headers of the received email. This has become a Bad Thing, because it means you send a bounce to a potentially (95%+ of the time these days) forged address, not to the original sender; this is collatoral spam.

The new way (probably not new as a configuration option, but certainly new as a "best practice") is to check the address and reject it during the SMTP connection. This tells the server actually attempting to deliver the mail to you that you don't want it, so nobody is involved who wasn't already a part of sending the invalid email. The original sender is still notified that the address they tried to send to doesn't exist, they're just notified by the other email server.

There are a few drawbacks. It used to be considered bad practice to give immediate feedback during SMTP as to whether an address exists at your server. I was taught early on that a mail server should disable the VRFY command, which asks "does this user exist"? The reason being, a script can connect to your mail server and randomly try different addresses in the hopes of discovering a working address to add to a spam-list. The new fix makes it pretty easy to do this, but it's pretty clear that these days most spammers won't bother to see if an address is legit, they'll mail to anything. (Spammers these days just don't have the class that the spammers did in the romantic old days, do they?)

Another problem is with mail servers that actually don't know what addresses are legitimate, such as those providing an email domain relay service, or acting as a gateway into some more complicated system. If you use a Unix mail server on the front end to block spam, and then deliver to an internal Exchange server, you may not want to have to keep user addresses synchronized. But if you don't, you're in danger of getting your mail server blacklisted for sending collatoral spam.

Ugly, ugly, ugly.

So how can you see which way your mail server is configured? Use your favorite terminal program to have this conversation:

kief@icarus:~$ telnet smtp.my.com 25
Trying 192.168.1.10...
Connected to smtp.my.com.
Escape character is '^]'.
220 smtp.my.com ESMTP Postfix
HELO icarus
250 smtp.my.com
MAIL FROM: myaddress@my.com
250 Ok
RCPT TO: nosuchaddress@my.com
550 : Recipient address rejected: User unknown in virtual alias table
quit
221 Bye
Connection closed by foreign host.

The above conversation means you're rejecting it right away. If it accepts the recipient, either that recipient exists, or the mail server is going to send a bounce to whatever address you enter in the Sender or From header in the body, if you continue the conversation.

Alternately, if your mail server is the same one used for SMTP delivery by your mail client, you can just use your client to create a message to a bogus address at your domain and try to send it, but this test depends on your client and configuration.

Map of the US indicating which term is popular in different parts of the country. I'm a "Coke" many myself - trademark dilution be damned, I'll never call it "pop". Found via Neil Gaiman.

Yesterday's NotCon was a winner. It was definitely a grass-roots geek affair, no corporate sponsors or goody bags, just something put together by a bunch of people who wanted to talk about and hear about personal projects various interesting geeks are working on. The start was delayed due to a "distributed denial of responsibility attack," as Danny O'Brien called it; "I was just doing what the wiki said."

Ewen Spence was another of the organizers, unmissable in his kilt. Check out his entry for details and links to other peoples' reactions, including plenty of pictures of rooms ful of geeks.

I enjoyed Danny's Life Hacks talk, which is supposedly going to eventually make its way onto a web site. This talk is basically "Habits of Highly Effective Unix Geeks", Danny has been reading lots of self help books of the organize yourself and get things done variety, and sent a survey to various prolific Unix hackers to find out how they organize themselves.

The social software panel was basically a bunch of guys doing stuff almost totally unrelated to social software, one guy basically plugged his Perl MVC framework (which could be used to build something like Orkut).

Music was a common themes, a lot of people were working on things related to distributing music.

The laptop distribution was interesting - Macs dominated, massively. There were probably about 80% Macs, 18% Linux, and 2% Windows, based on my eyeball survey. Dan was using a Mac which he had obviously moved to quite recently as he was fumbling around a good bit and taking shouted usage tips from the audience.

Danny, Dave, and the other organizers do a lot of things to help people use the Net for political activism, and there was a good panel on that. There was a talk on "Stalking your MP", which was basically encouraging people to start a blog for your MP if they don't have one already, and use it to browbeat them into starting their own site. the Public Whip puts voting records online, and the big new unveiling was They Work For You. This parses the record of Parliamentary debates which is published as a big, nearly unusable text file, and basically links it up so it's searchable, linkable, cross-referenced, feedable, and can be commented on. A very cool use of technology to open up the political process.

Test how Southern you are. It's a pretty rigorous test, I've lived in the South on and off since I was a kid, but not enough; I got 30 out of 71 questions. "Ranking: You need some real work, Yankee."

I'm looking forward to NotCon this Sunday, which looks to be a gathering of down to earth techie types doing interesting things with technology. "An informal, low-cost, one-day conference on things that technologies were perhaps not intended to do." Cool. No I won't be lugging my laptop and blogging it live, although it's possible there will be some sort of streaming video and/or IRC type action.