For people who have heard, I recently started a new position as a developer at ContextWeb. ContextWeb is a targeted/contextual advertising provider and exchange, and recently embarked on a technical re-architecture moving from C# to Java, moving from DBs to Hadoop, running a SCRUM/Agile environment with a bunch of pretty sharp people, free coffee, soft drinks, and a decent benefits package.

If you’re interested in working in downtown NY, I’ve linked in some of the open spots below, you can apply directly or forward your resume through me – let me know either way and I’ll put a good word in for you, maybe get you in to meet some people:

• Sr. Java Developer, Server Systems (working on my team)
• Sr. Java Developer, Data Processing/Hadoop
• Software Development Manager, Server Systems (be my boss)

Recently at work, had a need to rerun some load-testing numbers, but got stick since our internal servers all had builds we were looking at, or weren’t setup, or yada, yada… so we turned to EC2, with overall pretty positive results.

Some background – we build a web-based portal application that runs on a lightweight J2EE stack (Tomcat, Spring, Liferay) and do most of our testing w/jMeter. So, the process of setting up to loadteest on Amazon looked like:

• Build AMIs using Centos + our Applications (this took a bunch of hours, coming up to speed with Amazon’s tools for images – mostly the usual self-signed cert woes) — this was much easier since we have a one-click build for probably 80% of our full-machine footprint (that last 20% is a doozy though)
• Add a local LDAP server (OpenDS) to the image – was faster than troubleshooting some odd connectivity issues to our usual infrastructure
• Fire up some Amazon instances
• Grab one of the existing public AMIs w/Linux + Jmeter (Ubuntu in this case)
• Upload our scripts and start running

Net cost: ~ $40

Thoughts on the experience:

• Running jMeter inside EC2 is probably one of the things that makes this economically viable. Since our loadruns generate GBs, and GBs of traffic per minute in peak scenarious, paying per GB bandwidth can rapidly get expensive — but we were able to keep all the traffic in-cluster
• The fact that AMIs don’t by default have persistent storage (though EBS or new AWS features get around this – for more $$$) is actually a plus. We can run a test, shut down the image, bring it back up, and we have a clean image again to run to re-verify
• We were looking for stress/soak/smoke testing – so general benchmarks. For this need, we were not yet at the stage of verifying performance in a particular hardware or other environment, we just needed rough approximate numbers
• We chose not to use Amazon’s loadbalancing infrastructure, but might in the future
• EC2 actually let us run all our test scenarios (we have 3-4 main ones) in parallel, on spun-up instances – this cut our time to completion from something like 24-48 hours if we ran them serially to closer to 5hrs after setup was done – a big savings, especially for multiple runs)

All in all, it was positive experience, with some next steps we’re looking to incorporate:

1. Automate building AMIs – we did this by hand, and it was far and away the most time-expensive part (8 hours) and when we got errors early on we were immediately suspicious of configuration errors
2. Incorporate this into our Continuous Integration system (Hudson) – if construction of AMIs was automated (and Amazon’s tooling looks easy to automate) we could schedule this to run automatically, on a nightly/weekly basis – which would be nice, since right now this is a pretty heavily manual process – also, paying by the hour, the overall cost is a lot less than trying to manage with dedicated systems (and coordinate scheduling for the same)

I was recently asked via email what training or whatnot I would recommend to bring a programming team up to speed on Java.

A couple options come to mind – depending on the overall scope of your programming team and their past experience (if I can ask – what is your past experience?) but, I generally recommend the following progression of reading (note – this is probably about 6 mo of reading, if you like reading):

1. Pragmatic Programmer (non-language specific, but the best beginner book I’ve found)
2. Headfirst Java (for basic/core Java)
3. Effective Java, by Josh Block (Josh was the Architect of the Java Collection Framework)
4. Headfirst Design Patterns (I actually love this whole series)
5. Spring in Action (I think the latest edition was updated for 2.5)
6. Java Puzzlers (Josh Block, & I think Neil Gaffer)
7. Domain Driven Design, by Eric Evans

In terms of classes/training – general Java classes can be useful, but I find they generally tend to focus at too low of a syntactic/language level (understanding core concepts is important) – effective Java will give you a good feel for some of the oddities – but that your best bit tends to be to get enough to bootstrap you and start coding/testing/working.

Of course, my employer (CampusEAI) does provide focused training on topics like Java Development or Portlets, which since I had a hand in designing the modules I think hit the mark fairly nicely…

This may be old news to others, but I finally have my copy of the new “Sakai Courseware Management” book (courtesy of the folks over at Packt) and more surprisingly have even been able to carve out time to read the contents. For people who may not have been aware, this is the book that Alan Berg & Michael Korkuska have spent the last many months of their lives churning out.

After looking through “Sakai Courseware Management”, I’d say if you’re a technical staff member working with Sakai it’d be invaluable. Finally, much of the community knowledge and resources have been distilled into a single volume, greatly shortening the learning curve — and with enough topics that even old-Sakai hands will likely see some new bits, courtesy of the deep knowledge of Alan & Michael.

Continue Reading »

Google releases Wave protocol implementation source code – Ars Technica: “At the Google I/O conference earlier this year, the search giant revealed an intriguing new communication service called Wave that aims to deliver concurrent messaging and collaborative editing in a single cohesive environment. The underlying Wave Federation Protocol is designed to make it possible for third parties to host their own interoperable Wave instances. Google intends to open the source code of its own implementation in order to encourage widespread adoption of the protocol. The company took its first major steps in that direction on Friday by releasing the source code of its Operational Transform (OT) code and a simple client/server reference implementation that is built on top of the protocol. This code, which is available under the open source Apache Software License, will give developers a way to start experimenting with the protocol and potentially even building their own Wave-compatible services.”

(Via http://arstechnica.com.)

I hate to hop on the bandwagon, but I have to admit – Wave looks like the most revolutionary item I’ve seen in a while – in a full-on game changer sense. Not so much just because of the cool widgetry that Google’s built, but because it’s a protocol – with the flexibility and potential that implies.

Building on some of the interactions we’ve seen with IM, Blogging/Trackbacks, Twitter, and other messaging, Wave looks to standardize, federate, and embed real-time, multiparty communications to the point where it will become part of the fabric of the web. If Web 2.0 = comments and trackback conversations – this feels a lot more like Web 2.5 – the implementation we really wanted when we first tried to take the web from a document-based publishing platform to a conversation-enabled collaborative medium.

And… Open-Source production-quality reference implementation – what could be better. I have to say, not an small number of my off-work hours are going to be spent looking at embedding Wave into… everything… Particularly given that Federation (though still a little nebulous) is a first-class citizen in the platform.

We needed to populate some training course and class rosters into Sakai to let people play with site creation, roster attachment, and publishing tests to students. While we could have used webservices, we decided to leverage the built-in CmSyncJob and generate an XML file to dump out the data, and then sync it into the CMS (we wanted something easy to check into SVN for future usage).

The fun part was it let me use Ruby’s Builder (originally part of rails, but then extracted) to produce the XML, which was fun. Builder has a nice Ruby syntax that uses

to allow a nice language to XML mapping:

  puts "* Writing Academic Sessions..."

xml.tag!("academic-sessions") {
  2008.upto(2012) { |i|
    xml.tag!("academic-session") {
      xml.eid(i)
      xml.title("#{i-1}-#{i} School Year")
      xml.description("#{i-1}-#{i} School Year")
      xml.tag!("start-date","6/1/#{i-1}")
      xml.tag!("end-date", "8/31/#{i}")</pre>

And just run though a bunch of loops, and pretty clean method calls to spit out XML.

Thanks to http://blog.katipo.co.nz/?p=29 for the tips on Builder, especially the bit about using

to put in names which would translate to Ruby keywords (e.g.

where the - makes it

otherwise)

So… I was editing config.js to cut down the option from the default Sakai toolbar configuration for a Sakai build (which makes a huge diffence in being able to see stuff in the richTextAreas).

I got burned for a while (well, 10 minutes anyway) due to Firefox and some aggressive JS caching. Did a bunch of mvn deploys before I realized that was the problem.

Installed the Clear Cache Button which helped enormously.

Many thanks to Google, the FCKEditor docs, and the blogger who shared his pain (before me) about getting stuck due to Firefox’s caching.

ландшафт. the exhaustion that was the combined CampusEAI Portlets2008 and Annual conference is now behind us, and it seems like time for some reflections and observations. Hopefully some of these items will be items which I expand upon at a future date, but in no particular order, dumped straight from my brain:

• JSR-168 is here! Everyone really wants to write good standards compliant portlets. Architecture and engineering is a harder sell (or at least the time/cost trade-off) but there’s wide consensus that standard portlets are the way forward — at least excepting a couple of us widget fans
• SOA is something people are interested in, but that there’s been relatively little forward progress on. Some is governance. Some is tools (SOAP, WSDL, and what’s this REST thing?). Some is just that it’s big and strategic, and there’s many tactical must haves. I suspect some of it is also that much of our interesting data/services are locked in vendor platforms that have shown little interest in opening up. Though, a small trend does exist of creating SOA-style services to reach into vendor platforms and extract data from them
• Mobile wasn’t as big as I thought it would be. Not sure why. Most people seem to be interested in the abstract, but with few concrete plans. Maybe my iPhone has clouded my vision, but I do wonder if we’re going to get blindsided come fall — our target demographic is basically 18-22 year olds, afterall…
• AJAX in portlets is still hard. There are some tricks like wrapper divs, namespacing, and builtin support and integration patterns, but it’s still not a common practice.
• Identity Management is big. Governance is a big thorny issue, though many IT departments are rolling out vendor products from big players (Oracle, Sun, a little IBM) in the interim, tho ugh the exact scope of those items is somewhat unclear.
• Oracle is really putting portlets in lots of interesting places. Webcenter. Product mashups. Inside BI tools, and other GUI devices. I think they’ve probably embraced the architecture more than any other major vendor which is an interesting trend.
• Lots of awareness, and wanting to look at uPortal 3. Ooohs and ahhs over both the AJAX D&D, and maybe more importantly the new content adding UI — good going Jen!
• Really beautiful portals — some, though not all new portals really seem to be breaking out of the lots of boxes approach, or at least wrapping it in neat functionality like Boston College’s Agora design. Nifty trend. Sign of maturity?
• Community Development is hard. Aligning roadmaps, agreeing on implementation strategies, and putting all the pieces together is challenging. Even more so, justifying “doing it right” (and fit to share) versus quick and dirty, or getting a student up and running was a big trend. Makes my inner-engineer quail, but my inner-economist says that throw-away code lowers the barrier for solving problems, which is a good thing. Evolution isn’t always pretty and all that.
• Lots of desire for training, best practices, and advice on policy and governance. Real role for communities of practice, not just code and software.
• Increasing interest in “Enterprise Learning Management”. Lots of worries about migration, but the beginning of seeds wondering whether our current platforms are sufficient for a foundation for the next 10-15 years, and University strategic goals. Of course, some of this is the “enterprise IT guys” getting pulled into the LMS discussion for perhaps the first time in many places.
• Good beer is key to facilitating interesting non-session discussion. Content is king on the program, but largely only because it gets people in one place and produces interesting spontaneous interactions. Hands-on is something everyone wants, but it’s not clear a conference composed of many 1 hour sessions is the right format to deliver it.
• University IT teams wear many, many hats.
• British Universities seem to have a much richer and more abundent IT project management structure than (most) American schools. Really interesting thread about Imperial College in London blending ITIL and Agile methodologies.

I started this entry while I was riding a train back from the Laguardia ePortfolio Conference I’m endeavoring to reflect upon and synthesize threads from various (enlightening) presentations I’ve seen, and discussions I’ve had the privilege of participating in. First a brief plug: the content from the conference was fantastic — many congratulations to the folks from Laguardia Community College for organizing such a wonderful event.

Sakai has amazingly broad potential. The energy and excitement in the community and among those who have been watching Sakai make it clear that we’re really realizing the benefit of contributor’s blood, sweat, and tears in the form of some exciting tools for teaching and learning. Sakai seems uniquely positioned to become the base of a whole ecosystem of tools supporting different facets of the academic experience ranging from instruction, to assessment, to facilitating interactions between learners. I think we may be at a crossroads in terms of positioning, particular as we evolve towards explaining the product, beyond the project & community. Laguardia’s conference and discussions, especially those related to “Sakai vs. OSP” really focused my thinking on various opportunities for Sakai to support different areas of teaching, and learning.

A statement: I think the the common usage of Sakai to discuss both a specific set of tools supporting course/learning management (Sakai CMS/LMS?) and a platform/environment upon which those tools can be built and deployed has resulted in some confusion. I have heard many questions recently in the vein of “do you have to use Sakai to use OSP?” or “we’re a Blackboard school, and aren’t going to switch, does that mean OSP is out?” The fact that OSP is a toolkit built on top of Sakai (the platform) seems to be a confusing point for many who don’t currently have plans to deploy Sakai as a CMS/LMS.

To clarify, yes: it is quite reasonable to deploy OSP as a system exclusively dedicated to portfolios, completely separate from the other tools. Inputting text, adding reflections, uploading evidences, and managing assessment are all perfectly capable of being performed in a stand-alone environment. In the same way that past releases of Sakai downloaded from sakaiproject.org “stealthed” (hid) the portfolio tools an institution could choose to leverage the OSP piece of the Sakai ecosystem without forcing your users to adopt the entire environment — one advantage of the platform’s open-ness and customization capabilities.

In fact, I think this scenario illustrates a very real way to explain Sakai. If Sakai is a platform upon which bundles of tools (courseware, OSP, etc.) can be built, then we have a product with many facets. Each facet (LMS, OSP, Collaboration) supports a different interaction scenario, part of a greater whole of learning. Going forward, perhaps explicitly separating that greater platform from its concrete manifestations (particularly as courseware) would help emphasize Sakai’s potential as a learning suite or system — with facets focused on all aspects of a learner’s experience: courses, co-curricular’s, career advising, libraries & research, collaboration, and personal expression for a start. This thinking was really influenced by listening to many people talk about OSP — as a toolkit for building concrete artifacts: resumes, co-curricular transcripts, certification documents, personal expressions — all leveraging the same tools, but in many respects separate endeavors linked only by

I think there’s a danger that we could allow ourselves to slot Sakai into a box defined by the products that came before. though that’s where many adoptors initial exposure came from. The example of OSP illustrates the clear potential of Sakai’s modular architecture to enable assemblage of higher-level environments supporting particular styles of teaching or interaction. A common environment lets us both build on previous work, and also focus on integrating the experiences for out students and teachers, participants and leaders. My programmer’s mind sees this as being much the same potential as is now playing out in the Eclipse eco-system.

So the question I think this brings up is: if we focus on this broader picture, and think about these “bundles” as being the real deliverable, could we better frame this relationship by rebranding (consistent with recent thoughts about relaunching) the Sakai courseware tools as a separate entity within the Sakai umbrella — “Sakai Classrooms” maybe? Leaving room for thinking of the ecosystem as bundles, which you can mix and match: “Sakai Portfolios”, “Sakai Communities”, “Sakai Social Networking”. Different bundles of functionality built on the same platform, possibly using the same individual tools, but illustrating some of the broader possibilities.

Clay from Georgia Tech shot me an email recently which spurred me to try and put to words how my thinking has evolved about the relationship of an enterprise portal and Sakai, and where these technologies and communities are heading.

In general I think the focus of “enterprise portals” has always been one of integration and convenience, and as a result these products are moving towards being the place that knits together all the attention streams a user might have across the digital (and non-digital) campus. I think there’s a couple key use cases, some of which have more successfully been deployed than other.

1. One stop shopping (typical) + SSO
2. Summary Views & Aggregation

Less commonly actually implemented, though often talked about/pitched: 3. Dashboards 4. Actionable Intelligence (you have overdue books, return them!) 5. Deep aggregation (e.g. pulling in all the announcements from different systems and putting them into one stream)

In addition to portals focused on horizontal integration, I think we’re starting to see vertical integration around “portals” in Learning, Collaboration, HR/Admin, SIS, Libraries? and other clumps of functionality. Some of the goals around bundling related tools together are similar, but focused around a particular toolset, or context. At some point these could probably decompose into the “lots of tools/portlets in the uber-portal” that I think represented the portal thinking years ago, but I think the reality is market forces, as well as organizational and reporting structures make that unlikely to happen any time soon.

I suspect the interrelationship w a product like Sakai to a portal is mostly as a provider of information/data — pushing out items like announcements, scheduling, files in resources, and exposing them in a different context. Ideally if we shift our thinking more along the line of wire protocols (RSS, Real SOA, RESTful APIs) this I think positions us to also start doing “network integration” where Sakai can also start talking with and working with say Banner, or Kuali FS, or Facebook, or whatever platform. I’m very impressed with CARET’s mySakai work, and think John Norman’s vision on this is similar to the kind of plan I’d outline as benevolent dictator of the Sakai universe.

Along this line, I’ve scheduled another LMS-Portal integration BOF for JA-SIG and would like to use this project as the testbed for both a WG, and an incubated integration project within JA-SIG. I think a lot of the architectural level aspects should really span LMS’s — e.g. if we do it right, ANGEL, D2L, BB, and everyone should be able to use the same protocols, though Sakai seems an ideal reference implementation. I admit to being weak on knowledge of the IMS-spec side, and am not sure whether there’s work on that front we can leverage as well. So far what I’ve seen at least on the TI front has been less API/Data centric than I think we need to go though, though Enterprise seems promising.

One particular short-term item I’d like to see Sakai expose more broadly is the group contexts expressed in the form of class enrollments & particularly ad-hoc groups represented by project site membership. In many respects I think this is the most useful data in Sakai — it’s a social-network like context that integration with and hooking other systems into seems quite valuable. Enterprise grouping systems like Grouper while promising architecturally seems to have had slow adoption, and I suspect fitting systems like Sakai with something like OpenSocial or Google Contacts-like APIs to mesh groups together may get us farther faster in the short run.