all bits considered data to information to knowledge


Application Lifecycle Management is coming of age

Software intensive systems have a lot in common with humans – they are born, mature and die. They even sometimes come back from the dead, or simply linger around scaring day lights out of everyone who come in contact with them... To minimize one's chances of inadvertently releasing such monsters into the wild one should adopt holistic point of view – that of Application Lifecycle Management.

Wikipedia defines ALM as "... a continuous process of managing the life of an application through governance, development and maintenance. ALM is the marriage of business management to software engineering made possible by tools that facilitate and integrate requirements management, architecture, coding, testing, tracking, and release management."

Can't say it is a novel concept - every organization is already doing all of these either by design or by accident, with majority falling into the giant “in-between” void. The key is tight integration between three key areas – governance, development and operations.

To address the issue many a vendor came with ALM tools - sometime bundled, oftentimes integrated, - into a suite. Wikipedia lists over 40 “products” ranging from full-blown suites to assembly of specific tools, both commercial and free open source. Gartner's MarketScope mentions 20 leading vendors with ALM suites offering, out of which 8 got “Positive” rating, and IBM’s one got the only “Strong Positive”. The Forrester's Wave for ALM lists 7 vendors in the “strong” segment, with additional marks for market presence (with IBM, HP and Microsoft leading the “big” guys and CollabNet, Atlassian and Rally Software in leading smaller vendors pack)

The ALM offerings differ in degree of completeness, degree of coherence between the tools and extensibility model provided. Some of the more integrated offerings come in a variety of flavors such as SaaS or on-premises installations, with numerous options to complement either. And then there is a price tag to consider which, as with everything that purports to address enterprise-wide issues, is not insignificant – ranging from tens of thousand of dollars to a couple millions (and then some) with additional costs for infrastructure, operations, and maintenance. Still, there is a solid evidence that these investments under right circumstances might and do pay off. Implementing ALM principles to the Enterprise Integration and/or software development project can significantly improve quality of the delivered system and positively affect schedule.

The ALM processes fall into 5 domains:

  1. Requirements Definition and Management
  2. Quality and Build Management (including test Case management)
  3. Software Change and Configuration Management
  4. Process Frameworks and Methodology
  5. Integration Across Multiple AD (Application development) Tools

An integrated suite with a hefty price tag must address all of these domains to be worth consideration; and for best of breed route integration consideration are of paramount importance in order to realize ALM potential. One such important consideration, for example, is an integrated QA (either that or ability to integrate with a QA suite)

So far, only two vendors today offer fully integrated (all 5 domains), end-to-end, technology neutral ALM suites – IBM Rational Jazz Platform and HP ALM. The rest is either very much technology specific (such as Microsoft TFS) or stop short at providing some vital functionality (e.g. the issue and project tracking Jira does not address requirement management while FogBugz does, and neither comes close to providing test management functionality; both provide robust extensibility model to amend this with third party integration).  I am going to elaborate on the selection criteria and process of  “the best-fit ALM” solution in follow up posts. 🙂

Filed under: Uncategorized No Comments

BaaS: Books as a Service… and readers’ rights

Reading used to be a private experience… Walk along the shelves loaded with books, lightly touching the books’ covers, finding your prize, taking it cashier, bringing it home. Then you could read it in front of a fireplace,  or in the kitchen, or on a train, a subway, a lawn, a plane… If a storm knocks down a powerline you light the candles, and keep on reading. Alas, the days of unfettered access to your books are numbered, and I am not all that convinced that this change is for the good.

Just the other day, a company I took my training with had distributed its manuals through one of the ebook services that allow you to access material but prohibit you from printing or saving. Of course, you have to create an account, and agree to an umpteen pages unreadable legalese in which, in all likelihood, you give away your firstborn. The company reserves the right to collect data on you - your reading selection, your bookmarks, your reading habits (day, night, frequency), and offer you promotions based upon this information…. Now, don’t get me wrong - I’ve seen my share of the manuals weighing down on the shelves, unread, untouched, used as doorstops or BBQ fuel; and I am all for stopping the waste with replacing these with electronic copies. It takes some time to get used to but in the end we are all better for not having these almost-immediately-obsolete books around in hardcopy. Helps publishing companies bottom line, too, as it is significantly more difficult to pirate the content. Fair enough.

(But why do they have, for instance, “Fall of the House of Usher” by Edgar Allan Poe advertized for just $0.99?! It has been around since 1839, and went through numerous prints, and is available  for free from the internet (Project Gutenberg site be but one example)…

But distributing contemporary titles through this channel has even more sinister overtones  - the Big Brother watching (potentially) your preferences of politics, literary tastes, drawing conclusions about your leanings, preferences, demographics… I can live with electronic media as long as I own what I’ve bought, even with the limitations imposed by the publisher, but I don’t want to surrender my privacy for the sake of dubious convenience! (or, at the very least, I want to make an informed decision about it)

It seems all but certain that the printed books will gradually become luxury items, with prices to match. The mainstream books will become a service: pulp fiction will be read exclusively on iPads/Phones, Kindles, Kobos and other electronic devices; the college textbooks will be rented electronically, and so on… I see it all but inevitable. But we - the readers - must not sell our birthright for the convenience of “bread and some lentil stew”.

We must insist on privacy provisions, similar to those afforded to credit card holders, on equivalents of "do not track" and "private reading" options found in web browsers. This is our birthright as readers.


A glimpse of future: Just-In-Time Information

Ever-shrinking attention span of the younger generation gets quite a bit of attention (pun intended) from the researchers and educators (e.g. "How Social Media Is Ruining Our Lives" - over the course of the last ten years the average attention span has dropped from 12 minutes to a staggeringly short 5 minutes )

Yet I wonder. Maybe we do not need long attention span in the era of informational deluge pouring through smartphones, tablets and laptops? The pervasive nature of internet is changing the way we collect and process information. No longer do we need to own information, we only need to know where/how to find it, and how to connect it with other bits we've already found.

Memorizing information was the staple of a rote learning for centuries - people traveled to read a copy of the book in particular library or listen to particular lecture; movable type and audio/video recording changed this - books/records/movies become more readily available, in a library or purchased from a bookstore. As time passed, books became ever more affordable - but they still were self contained: the information in a book/magazine/movie was distilled and structured to provide all the components needed. With the advent of Internet and electronic media this began to change - it became possible to transform raw data into information just in time. And the premium is not on ownership but on speed of finding and processing the data, ability to evaluate and integrate it on-the-fly, and - what's the word- the critical thinking.


DEC: Sic transit gloria mundi

I haven't given much thought to DEC/Digital ever since I had interviewed with the company back in 1990s (no, didn't get the job); then a brief schadenfreude moment  in 1998 when I've learned the company was sold to Compaq (which, incidentally, I was consulting for at the time), and then once more - when a (t)rusty VAX/VMS system was migrated to Solaris in yet another company I used to work for..

Until an article has been brought recently to my attention that summarizes the company's path in a single page, with a few links to DEC PDP manuals (hosted, ironically, by Microsoft research) as well as where more comprehensive information can be found. The company had pioneered many a breakthrough in technology before fading into oblivion during what was arguably the golden age of the Internet - roaring 90s...

Historians have been pondering on what makes for a long-lived organization - be it a business entity, political or religious one - since the beginning of the time, always armed with 20/20 hindsight vision. Yet the formula remains elusive.

What was it: close-mindedness? laurel-resting or "too-busy-sawing" syndromes? ossified management? all of the above?

Victot Hugo is often quoted about power of "the idea whose time has come"; is there a penalty exacted for the ideas that were ahead of their times? Like Digg (a more dramatic fall than even Friendster or MySpace) who pioneered most of the Facebook concepts - the latter has valuation upward of 100 billions while the former was just recently sold for $500K (likely just a tad above costs of its office furniture and computers)...

Tagged as: , 2 Comments

Retina tracking BI: consent is not required

Business Intelligence quickly moves past analyzing our conscious responses... it is after what you really think - not what you report thinking.  Forget polls and questionnaires - the best data is collected from the subjects not even aware of the process. Take the heat-map generated by Unilever analyzing shoppers eye movements... or compare designs produced in0house with these vetted by consumers subconsciously preferring one shape over another...

No consent required - the very fact that you stepped into the store (or visited an online site)  implies that the retailer is free to track your every movement, or ambush you with colors, music or odors designed to induce specific behavior.  I could easily imagine a system tracking reaction on images of politicians, say, Romney and Obama; I bet it would be a much better indication of voting patterns than old-fashioned door-to-door pollsters - and also would open a giant can of worms on invasion of privacy issues...

Where  do we draw the line in legitimate use of data?

Tagged as: , , No Comments

I know what you read last summer… And I know what you’re reading now

With proliferation of electronic reading devices we surrender many personal liberties we've taken for granted for so long: now it is possible not only to find what and when you bought a book but also whether you've read it, for how long, on what days of week, at what time, what drew your attention... As Wall Street Journal's article puts it "Your E-Book is reading You".

Convenience comes with many strings attached though. What would electronic equivalent of Bradbury's Farenheit 451 look like? The entire messy business of replacing hard-copy of newspapers and books detailed in Orwell's 1984 went away replaced by infinitely malleable bits and bytes. Nobody misses developing films - what about times when a photographic negative was a considered an irrefutable proof?

Personal reading experience becomes a raw material for data analysis, and I, for one, am rather uneasy with this brave new world. This adds to yet another piece of puzzle for constructing your personality on social networks, where people and organizations with BI savvy are mining your personal experiences in hopes to sell you ever more stuff (e.g. How Target Figured Out A Teen Girl Was Pregnant Before Her Father Did) - or for some other, not always as benign, reasons.



Automated translation on shoestring budget

The snail mail deluge have abated in recent years, and yet once in awhile an officially looking piece of a dead tree documents finds its way in my dad’s mailbox. Herein lies the problem - my dad’s English skills are rudimentary at best, and he cannot distinguish between important stuff and junk mail easily; additional constraints - fear of computers (which precludes "advanced" operations such as copy'n'paste into A solution I eventually came up with was to set up an automated translation system.
The whole setup was surprisingly easy to implement, though certain assumptions had to be made. Here are the prerequisites for the particular solution I’ve devised for him:

  • Fast internet connection
  • Windows XP/Vista/7 based computer
  • Microsoft Office (specifically Microsoft Word) 2003 or later
  • A scanner

Assuming that all the above requirements are met the following instructions would get you to translate printed documents in no time.

  1. Download and install FreeOCR software here
  2. Configure the software to recognize your scanner (it has to be already installed, connected and turned on)
  3. Place document into the scanner, and click [Scan] button on the FreeOCR toolbar; the scanner should be set to scan at 200dpi or better (normally, 300dpi)
  4. Once the scanned text appears in the left-hand pane, make sure that it is aligned properly so you ca; use buttons on the left vertical toolbar
  5. Press [OCR] button - depending on the length of the original scanned document it might take some time to convert all of it into editable text.
  6. 6. The next step involves saving the produced text as MS Word document; the Microsoft Word 2003 or later must be already installed on the computer for this to work. You can also at this point copy the text to clipboard and use it with alternative translators such as Google Translate; here I am using Microsoft built-in functionality to minimize number of steps required.
  7. Once the FreeOCR transfers the text into a newly opened instance of the Microsoft Word, you could go and correct some text that did not come out well with the OCR process (you can also do it within FreeOCR text pane but it is much nicer to have full support of spellchecker)
  8. 8. Now you are ready to translate the document into a language of your choice using Microsoft Online Translator. Word 2007 and 2010 have this capability built-in, and MS Word 2003 can add it with a download from Microsoft
  9. 9. From the right-hand pane you can select choice of both the source and the target languages, and click on green button labeled with “Translate the whole document”
  10. The program will as you whether you consent to sending the information over the internet…your call.
  11. The translation pops up in the Internet Explorer browser (even though you might have a different one defined for the system)
  12. Done. At this point you can print the document, save it or copy and paste into different applications.

The proposed system relies on Microsoft Translator remaining free in the future, and FreeOCR staying free. These components could be substituted with different commercial and non-commercial options for scanning, optical character recognition and translation.


Ethical limits of Business Intelligence

Intelligence of all kinds can be gleaned from the mounds of data accumulated from our daily interactions with the outside world such as business intelligence or social intelligence. It then can be used to manipulate our behavior to the benefit of the data collector/analyst.

Here is, for example,  how IKEA and Costco utilize information "to turn browsers into buyers, and making buyers to spend more". A new layout of the store floor or combination of sounds/lights/olfactory stimuli to put us in "buying mode", targeted advertising, mass customization based upon data collected from purchasing history, Facebook, LinkedIn, Google+... For example:

"In research yet to be published, a University of Alberta team has proven that what we smell and hear affects what we buy: When a sample group smelled the relaxing scent of lavender, 77% wanted a soothing iced tea, but when the same group smelled the arousing aroma of grapefruit, 70% reached for an energy drink. When the researchers played Mozart’s Sonata in D Major at a slow tempo, 71% wanted iced tea, but when the piano piece was sped up, 71% wanted an energy drink — an exact reversal."

Where does "legitimate use" stop and "Brave New World"/"1984" take over?

Where is this limit after which these "insights into consumers" behavior become invasion of privacy?



Lots of little brothers… all watching you

Predictive analytics at its best... and worst.  Charles Duhigg's article How Companies Learn Your Secrets published in New York Times opens a big can of worms here. The truth is that we are getting better and better with predictive analysis aided by ever powerful computers and software, and better mathematical models... and we are getting closer to the point where our secrets do not even have to be stolen as they could be inferred from mountains of tiny clues we left behind as we are going after our daily lives.

The key to make this happen, the facilitator is unique identifiers we acquire with our credit cards, loyalty cards and other numbers that could be used to track your activities. It has its uses - such as prevent fraud, prepare for an eventual disaster and so on.. But there is more insidious side to the predictive nalytics - instead of Big Brother watching we have hundreds of small ones actively engaged into collecting and trading our personally identifiable information - something we are only too happy to give away for a few pennies in discounts on overpriced merchandise.  So goes our privacy - not with a bang but with a whimper



Walking the dog on Facebook

For over a year now I was observing a puzzling behavior of my friends and relatives who spend considerable amount of their free time socializing on Facebook - posting bits of news, pictures, responding to posts of their friends, liking and "unliking" their messages, tagging their photographs etc...

The epiphany came while I was walking my schnauzer through the neighborhood lawns and bushes: the semblance was uncanny - scout the grounds, check messages left by other dogs, mark the spot (I can only surmise whether this activity means creating a new message of his own, or merely responding to somebody else's message) ...

Could be that obsessive socializing is an inherent trait of all sentient beings?

Full disclosure: I also have a Facebook account... don't use it much though. 🙂