all bits considered data to information to knowledge

29Sep/118

NTLM authentication with JMeter (sort of)

I have been using JMeter for quite some time now, and consider it to be an exteremely useful tool for all kinds of testing - especially web apps. It is not as polished as some commercial apps out there but by virtue of being an open source (and free!) it proved to be quite adequate for my team. Until we bumped our heads againtst NTLM authentication required by Sharepoint 2007 server.
Following the documentation  [HTTP Authorization Manager] was added and configured all the properties as described in the documentation. Yet no matter what we've tried, we were always getting 401 response - "Not authorized"; quick Internet search confirmed that I am not the only one struggling with this problem yet somehow solutions proposed did not work in my particular situation.

Digging into the source code (gotta love open source!) I've found that it fails NTLN challenge/response hoops (see this link for a very detailed explanation on how NTLM works), and then the following information posted on Apache Foundation site regarding use of NTLM with version 4.1.2 HTTP Components (used by the JMeter 2.5 version we are using)
Turns out that "there are still known compatibility issues with newer Microsoft products as the default NTLM engine implementation is still relatively new"... and the maintainer of the code put together a quick workaround to show how to use "more established and mature NTLM engine developed by Samba project."

The article does a great jobs showing details of implementation (along with the reasons why it is not part of HTTPClient library) but stops short of providing a working example, which is the purpose of this post.

Disclaimer: this is but a quick'n'dirty proof of concept (hardcoded values, console outputs, no unit tests or logging etc);   the sole intention of this code is to illustrate the concept.

The project contains two source packages, one for NTLMEngine and NTLMSchemeFactory  interfaces, and one - NTLM_ping - providing the main executable which imports JCIFSEngine.NTLMSchemeFactory.

The structure of the project (including depensdency JAR(s)) is shown on the picture below

 

 

 

 

 

 

 

 

 

and here are the results of successful execution


 

 

Finally, the source code for NTLM_ping.java ( the contents of the JCIFSEngine.java and NTLMSchemeFactory files are provided at the Apache Foundation post I've mentioned above).

------------------------------------------------------------------------------------------------

package NTLM_ping;

import org.apache.http.auth.AuthScope;
import org.apache.http.auth.NTCredentials;
import org.apache.http.impl.client.DefaultHttpClient;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.HttpResponse;
import org.apache.http.client.params.CookiePolicy;

import JCIFSEngine.NTLMSchemeFactory;

public class NTLM_ping {
 public NTLM_ping() {
     super();
     }

 public static void main(String[] args) throws Exception {

        DefaultHttpClient httpclient = new DefaultHttpClient();
        httpclient.getAuthSchemes().register("ntlm", new NTLMSchemeFactory());

        //add credentials
        httpclient.getCredentialsProvider().setCredentials(
            new AuthScope("host", -1),
            new NTCredentials(
                      "user"
                    , "password"
                    , "host"
                    , "domain"));

        HttpGet httpget = new HttpGet("http://<url>");
        //ignore cookies
        httpget.getParams().setParameter("http.protocol.cookie-policy", CookiePolicy.IGNORE_COOKIES);
     try {
     // execute the GET
            HttpResponse status = httpclient.execute(httpget);
            System.out.println(status.getProtocolVersion());
            System.out.println(status.getStatusLine().getStatusCode());
            System.out.println(status.getStatusLine().getReasonPhrase());
            System.out.println(status.getStatusLine().toString());
     } finally {
     // release any resources
        }
    }
}

------------------------------------------------------------------------------------------

It would be relatively simple to wrap this code as a custom element/sampler for JMeter though one would have to pay attention to licensing issues (JMeter is licensed under Apache licennse, while JCIFS Samba libraries are under LGPL)

A very detailed tutorial by Mike Stover and Peter Lin on extending JMeter w/plugins can be found at Jakarta website.

15Dec/092

Speeding up Sharepoint 32bit development environment

Sharepoint 2007 (MOSS) requires Windows server platform to run on (yes, I am aware of Bamboo hack). This would require either set up a central server for the developers, or equip the developers with Windows 2003 Server machines, or resort to virtualization.

 

This is the latter one that I am talking about speeding up. The 32bit  Windows environments have a hard limit , being able to address only up to 4GB of physical memory

 If you are using virtualization to run Windows 2003 Server on Windows XP 32bit your machine slows to a crawl. If for some reasons (and I have tons of those right now) you cannot switch to 64bit development environment, you can go for 64bit HOST environment (Windows XP or 7), and run 32bit virtual GUEST development environment (Windows 2003 Server). 

 Using VMWare Server (free) you’d be able to allocate up to 3600MB of RAM just to your virtual machine( and leave some to the host syste, say, another 4GB). My non-scientific profiling of 64bit Windows 7 (6GB of RAM, total) running 32bit virtual Windows 2003 Server shows improvement in performance by factor of 10 (in some operations) against 32bit Windows XP running 32bit virtual Windows 2003 Server.

 To sum it up: if you must do 32bit Sharepoint development, do it in 64bit host environment to work around 4GB of RAM limitation (read: incessant paging).

7Dec/092

Sharepoint Linkapalooza…

2010

A rather thorough overview of the upcoming Sharepoint 2010 from ComputerWorld

The official site: 

http://sharepoint2010.microsoft.com/Pages/default.aspx

developers resources: 

http://sharepoint2010.microsoft.com/product/Benefits/IT-Developers/Pages/default.aspx

and the rest (euphemistically called "IT Professionals"): 

http://sharepoint2010.microsoft.com/product/Benefits/IT-Professionals/Pages/Top-Features.aspx

Microsoft Sharepoint Conference 2009 (Las Vegas): 

 http://www.mssharepointconference.com/Pages/default.aspx

Getting Started with SharePoint 2010

Get Started Developing on SharePoint 2010

Learning Snack for Developers: Developing Solutions with Microsoft SharePoint 2010

Sharepoint 2007:

A brief overview of Sharepoint workflow options (2007): Share Pointers: Selecting the Right Workflow Technology (Bjorn Furuknap)

2007

Case Studies

Coca-Cola Enterprises

Kraft Foods

Hawaiian Airlines

Optimization

Microsoft Enterprise Content Management (ECM) Team Blog 

Microsoft SharePoint Team Blog
http://blogs.msdn.com/sharepoint/
http://blogs.technet.com/wbaer/archive/2009/12/06/configuring-the-web-and-application-server-roles-for-microsoft-sharepoint-server-and-foundation-2010.aspx

ESRI MAPIT for Sharepoint
http://www.esri.com/software/mapit/evaluate.html

8Nov/090

Software mega-packages in the SOA world

What do you get when you add content management, eForms, collaboration, messaging and workflows – basic building blocks of ECM (Enterprise Content Management) suite such as MS Sharepoint, Adobe LiveCycle/Alfresco etc.? A platform for assembling a custom ERP (Enterprise Resource Planning) application.

For decades SAP had an iron grip on the ERP market (with ~30% of the total ERP market and ~25% of SCM and CRM markets) ; also-runs Siebel, Lawson, MS Dynamics etc dividing remaining 60% of the market. With advent of ECM one could connect proprietary and/or open source/free components into a powerful ERP app for a company that isn’t prepared to spend millions on SAP, and adopt their “best practices” in the process; instead they could implement their own best practices and maybe even re-use their existing accounting, planning or material management software.

Maybe, the era of mega packages is over…

7Nov/090

In the beginning there was HTML…

In the beginning there was HTML, and saw Marc Andreessen that HTML was good, and created Netscape browser. And while the browser was buggy and slow, it changed the world, and begat (spiritually) Firefox, Safari, IE, Opera, Chrome and others. Somewhere along the road geeks were expelled from paradise, and business moved in to cash on the Internet…

It  used to be that one could make a living coding HTML by hand, and a number of companies were created to automate the task – injecting standard headers, visualizing website structure, propagating changes, editing JavaScript etc.

Meanwhile, dynamic content generation went from CGI scripts (Unix shells, C, Perl etc), through server-side scripting languages, all the way to ASP, PHP and the likes…Java made its debut in the web via applets, followed by Microsoft knee-jerk reaction to embed ActiveX controls in the IE (and opening the floodgates for security threats in the process), before finding its way to servlets/portlets and other *lets, and Microsoft counterparts – asp.net (with code behind), web controls and webparts…

At every step the community opted for ever more tightly controlled development (naming conventions, standardized logging, error handling, continuous integration), ever greater abstraction level (reusable libraries, IDE, tools, patterns, frameworks), ever greater automation of every stage of software development – from coding to building to testing to deployment to maintenance… Enterprise level software moved from monolithic tightly coupled monstrosities to distributed component based systems, from closely coupled binary contracting to loosely coupled Service Oriented architectures…

I believe, we are entering the phase when ECM – Enterprise Content Management systems - are becoming platforms within which data and applications come together, where composability of the components allows for stringing functionality in ad-hoc applications that are enabled by all of the above: technology independent, loosely coupled, location agnostic nirvana!…

Now, back to Earth. All these concepts have to live somewhere. A number of ECM sprouted in recent time: every heavy weight (Microsoft, Adobe, IBM, Oracle), and upstarts (Alfresco, for instance). None is perfect, all have dizzying array of features, are based on different technologies (.Net, Java, Flash/C etc); arcane programming models…
Yet, I’d argue that they are step in the direction I’ve just plotted in my ramblings above, and are here to stay. This is just a beginning where all these trend converge to produce a qualitative leap forward; one can tell by how the forerunners are trying to subvert it by fencing you in, trying to be everything at once… But the ship has already sailed, ultimate distributed computing (SOA) found its frameworks to move away from craftsmanship into industrial age of software development.

22Oct/090

Sharepoint 2010: prayers answered… sort of

The  SharePoint  2010 is coming!.. In fact, it is almost here – the rumour has is that beta was already released to MSDN subscribers last Wednesday (10/21/2009, that is). Not a revolutionary upgrade in a sense that SP2007 was to SP2003, but a major milestone nevertheless.

The interest in the product speaks for itself - the conference 2009 was sold out - and that's 7,000+ attendees! I was a bit surprised to see that many people  from Europe - for some reason there were Danes at every turn - but I've also met folks from France, Germany, England, Spain, Czech Republic, Netherlands, Belgium, Finland, Israel... there were quite few Kiwis and Aussies, Brazilians and Argentineans..

Having inherited a team of hard core Java developers I am facing quite a challenge to help them in transition to proficient Sharepoint ones. I wish I could fly all of them to Las Vegas, just to feel the energy in the room, attend all sponsored events (yes, free drinks included), and see elegancy of some solutions! This would be the best way to assure them of an interesting and profitable career path... Damn these budget cuts!

Apparently, Microsoft heard the developers community, or, maybe, lusted after the fat profits that third party vendors made patching the holes left in the MOSS 2007... But the biggest improvements for me came from the developer's perspective. The biggest gripe about Sharepoint development came from the fact that it is not a development environment. It is very painful to code against, debug and deploy in Sharepoint 2007, not to mention absence of testing framework (there are third party tools, to be sure), and then this insistence on using VSTS... (no, you do not need it – we are developing with VS2008 Professional). I am happy to report that SP2010 and Visual Studio 2010 have addressed this problem. Beginning from 2010, one can develop decent workfl0ws in Sharepoint 2010, and still maintain sanity (most of the time). First, you can develop on Windows 7 or Windows Vista SP1 (64 bit only, of course); no need for sluggish clunky VM that hogging down your computer (btw: you can boot to VHD, if you'd like to keep your SP development separate from the rest of your work). Second: no more add-in packages to do development in Visual Studio , no multiple deployment steps to install and activate provisioning code - it all built in; no SDK, no missing references.. Finally:  you can deploy from within Visual Studio (God, I missed this one!), and you can debug your stuff without jumping through additional hoops.

A big thing: WSP now can be created in SharePoint Designer (still free), and edited  in Visual Studio (and vice versa). Finally.

BDC are now BDS - business data services, and are better than ever. You can combine data from different data sources into a single consistent interface. Oh, and don't forget the ability to create EXTERNAL lists. Sharepoint 2010 is all about RESTful services, and json; a novel way to provide and consume data; ability to create lists with multiple columns is very welcomed, too.

I am very intrigued by Access Services in Sharepoint 2010; I see how it can help me to track down every pesky MS Access app crawling in the enterprise, and manage them centrally. Very cool (more about it later).

LINQ is staging a comeback. After hinting that it will be deprecated in favor of Entity Framework, Microsoft again puts it at the heart of data interfaces. Virtually every demo that made use of data did it through LINQ; and it was prominently displayed on the slides presented by the m'softies. Something to think about.

All in all, it appears that our prayers (and curses) has been heard.