Sunday, August 24, 2014

Web APIs and API Management


It's a world of APIs. In the beginning only the big technology firms like Google, Twitter that catered to large developer community and mobile apps needed to expose APIs. But now, there can pretty much be no business with out APIs exposed to the outside. There are many reasons a business would want to expose APIs mobile apps being one of the foremost reason. Businesses would want make it easy for their partners, customers, vendors etc. by automating data exchange or transactions. With the advent of IOT APIs are going to become ever more important (machine to machine interaction).

Building RESTful APIs has become extremely easy with most technologies providing inherent frameworks. With ASP.NET Web API .NET developers can convert their old web services or applications into RESTful API in  matter of days if not hours. Similarly Jersey and Restlet among others make it easy to develop WebAPIs in Java. Modern languages such as Ruby on Rails and NodeJS are of course meant to be modern and cater towards WebAPIs and web application development. There are several books on RESTful API design which I believe is very important for technical leads and architects to refer to before building APIs for their organization.

The next step after designing and building the APIs is the API management functionality. We need the ability to manage the developers, monitor and track their activities and control their access. While this functionality can build in house or use of the several API management solution companies. There are many different "types" of providers in this space. One of the popular type of API management is the cloud hosted API gateways. Others types are on-premise software, on-premise appliance. The one I like and implemented is a plug-in or agent based solution which makes it a hybrid on-premise/cloud architecture.

Cloud Solution: Major vendors such as Layer 7, Apigee, Mashery etc. all offer cloud deployment solutions. In this model the cloud API gateway acts as a proxy to the APIs in your data center. API Management aspects such as developer on-boarding, key management, throttling, billing (if you are going to charge for your APIs) are all handled by the solution provider in the cloud. APIs may still stay in your data center. The provider makes the call to the API on behalf of the requests coming from developer applications. Some of the advantages of cloud solutions are easy setup and fast time to market. Some concerns are security and network latency.

On-Premise Solution: Again several of the major vendors also provide an on-premise solution. In this case vendor provides software or an appliance that you need to setup in your data center. The software or appliance will act as a proxy to the underlying WebAPIs. Some advantages are internal only situations, security and network latency. Some disadvantages are additional burden on IT to support hardware/software and cost.

Hybrid Plug-in solution: In this model the requests are made directly to the APIs. A plugin or agent intercepts the calls and checks with cloud based API management application (through API calls of course) before servicing the request. This solution provides ultimate flexibility and takes care of both internal and external situations. In addition to being a low cost solution, since this is a plug in model it is easy to replace if ever needed to replace with a different solution. 3Scale provides this kind of solution. There may be other vendors (Mashery?) providing similar solution, but I am not familiar.

Who are the players in API Management

There were not many players in this space only a couple of years ago. Now it seems like every technology company is a player in this space! Here is a small list:
Apigee
Layer 7
Mashery
3Scale
Oracle
Microsoft
IBM
MuleSoft

Open Source software:
WSO2
ApiAxle
API-Umbrella


Tuesday, March 25, 2014

Startup Shutdown scripts for WebLogic on Windows

It's a serious pain in the neck to setup WebLogic (11g at the moment) to automatically stop and start along with the system startup/Shutdown. Oracle should seriously consider doing something about it. Ideally install as Windows Service as soon as a domain is configured. However until that happens we are on our own.

I search around the web if someone had created an ideal solution already a script/document that I could follow and get this going in matter of minutes but alas I couldn't find a complete solution. I found bits and pieces here and there. So I took them and tailored them to write this blog. So here are the steps (this document assumes you have already configured node manager as a service and it is running)


1. Make sure Node Manager credentials are set correctly in WebLogic Admin Console

Go to the WebLogic console in Domain Structure, click on Domain Name - top level (e.g. oam_domain), click on Security tab and under General sub-tab click Advanced. Set the Node Manger credentials here (e.g. weblogic / Password123)

2. Add those credentials are put in nm_password.properties

Find (or create) the nm_password.properties file under \user_projects\domains\\config\nodemanager (e.g. - D:\Oracle\Middleware\user_projects\domains\oam_domain\config\nodemanager). Add the Node Manager credentials to the file. e.g.-

username=weblogic
password=Password123

3. Create boot.properties for the Administration Server


Follow these steps to create the boot.properties file:

Go to \user_projects\domains\\servers\AdminServer\security directory. e.g.-

D:\Oracle\Middleware\user_projects\domains\oam_domain/servers/AdminServer/security/

Use a text editor to create a file called boot.properties under the security directory. Enter the admin credentials. e.g-

username=weblogic
password=Password123

4. The scripts

A. weblogic.properties - Properties file used by scripts:

DOMAIN_NAME=oam_domain
MW_HOME=D:\\Oracle\\Middleware
DOMAIN_HOME=D:\\Oracle\\Middleware\\user_projects\\domains\\oam_domain
WEBLOGIC_USER=weblogic
WEBLOGIC_PASSWORD=Password123
HOST=myweblogicserver.company.com
NODEMANAGER_PORT=5556
ADMINSERVER_NAME=AdminServer
ADMINSERVER_PORT=7001
ADMINSERVER_URL=t3://myweblogicserver.company.com7001
MANAGEDSERVERS=oam_server,oim_server

B. startAdminServer.py - Starts the Admin Server

from java.io import FileInputStream
import java.lang
import os
import string
import datetime

logfile = "startManagedServer.log"
LOGFILE = open(logfile,"a")
now = datetime.datetime.now()
LOGFILE.writelines("Starting Admin Server - " + str(now) + "\n")

propInputStream = FileInputStream("weblogic.properties")
configProps = Properties()
configProps.load(propInputStream)

WEBLOGIC_USER = configProps.get("WEBLOGIC_USER")
WEBLOGIC_PASSWORD = configProps.get("WEBLOGIC_PASSWORD")
HOST = configProps.get("HOST")
NODEMANAGER_PORT = configProps.get("NODEMANAGER_PORT")
DOMAIN_NAME = configProps.get("DOMAIN_NAME")
DOMAIN_HOME = configProps.get("DOMAIN_HOME")
ADMINSERVER_NAME = configProps.get("ADMINSERVER_NAME")
ADMINSERVER_URL = configProps.get("ADMINSERVER_URL")

nmConnect(WEBLOGIC_USER, WEBLOGIC_PASSWORD, HOST, NODEMANAGER_PORT, DOMAIN_NAME, DOMAIN_HOME)
LOGFILE.writelines("Connected to NODE MANAGER Successfully...!!!" + "\n")
print ''
print '============================================='
print 'Connected to NODE MANAGER Successfully...!!!'
print '============================================='
print ''

print '###### ADMINSERVER NAME = ', ADMINSERVER_NAME
nmStart(ADMINSERVER_NAME)
LOGFILE.writelines("Successfully started " + ADMINSERVER_NAME + "\n\n")
print ''
print '============================================='
print '===> Successfully started ', ADMINSERVER_NAME, '  <==='
print '============================================='
print ''

C. stopAdminServer.py - Stops the Admin Server

from java.io import FileInputStream
import java.lang
import os
import string
import datetime

logfile = "stopAdminServer.log"
LOGFILE = open(logfile,"a")
now = datetime.datetime.now()
LOGFILE.writelines("Stopping Admin Server - " + str(now) + "\n")

propInputStream = FileInputStream("weblogic.properties")
configProps = Properties()
configProps.load(propInputStream)

WEBLOGIC_USER = configProps.get("WEBLOGIC_USER")
WEBLOGIC_PASSWORD = configProps.get("WEBLOGIC_PASSWORD")
HOST = configProps.get("HOST")
NODEMANAGER_PORT = configProps.get("NODEMANAGER_PORT")
DOMAIN_NAME = configProps.get("DOMAIN_NAME")
DOMAIN_HOME = configProps.get("DOMAIN_HOME")
ADMINSERVER_NAME = configProps.get("ADMINSERVER_NAME")
ADMINSERVER_URL = configProps.get("ADMINSERVER_URL")

connect(WEBLOGIC_USER, WEBLOGIC_PASSWORD, ADMINSERVER_URL) 
LOGFILE.writelines("Connected to Admin Server Successfully...!!!" + "\n")
print ''
print '============================================='
print 'Connected to Admin Server Successfully...!!!'
print '============================================='
print ''

print '###### ADMINSERVER NAME = ', ADMINSERVER_NAME
shutdown(force='true')
now = datetime.datetime.now()
LOGFILE.writelines("Successfully stopped " + ADMINSERVER_NAME + " " + str(now) + "\n\n")
print ''
print '============================================='
print '===> Successfully stopped ', ADMINSERVER_NAME, '  <==='
print '============================================='
print ''

LOGFILE.close()

D. startManagedServers.py - Starts all the Managed Servers (comma separated list in properties file)

from java.io import FileInputStream
import java.lang
import os
import string
import datetime

logfile = "startManagedServer.log"
LOGFILE = open(logfile,"a")
now = datetime.datetime.now()
LOGFILE.writelines("Starting Managed Server - " + str(now) + "\n")

propInputStream = FileInputStream("weblogic.properties")
configProps = Properties()
configProps.load(propInputStream)

WEBLOGIC_USER = configProps.get("WEBLOGIC_USER")
WEBLOGIC_PASSWORD = configProps.get("WEBLOGIC_PASSWORD")
HOST = configProps.get("HOST")
NODEMANAGER_PORT = configProps.get("NODEMANAGER_PORT")
DOMAIN_NAME = configProps.get("DOMAIN_NAME")
DOMAIN_HOME = configProps.get("DOMAIN_HOME")
ADMINSERVER_NAME = configProps.get("ADMINSERVER_NAME")
ADMINSERVER_URL = configProps.get("ADMINSERVER_URL")
MANAGEDSERVERS = configProps.get("MANAGEDSERVERS")

while True:
try:
print 'Trying to connect to AdminServer...'
connect(WEBLOGIC_USER, WEBLOGIC_PASSWORD, ADMINSERVER_URL) 
break
except:
print 'Could not connect to Admin Server. Sleeping for 30 seconds...'
sleep(30)

LOGFILE.writelines("Connected to Admin Server Successfully...!!!" + "\n\n")
print ''
print '============================================='
print 'Connected to Admin Server Successfully...!!!'
print '============================================='
print ''

MANAGEDSERVERLIST = MANAGEDSERVERS.split(',')
for MANAGEDSERVER in MANAGEDSERVERLIST:
print 'Starting managed server...'
print '###### MANAGEDSERVER = ', MANAGEDSERVER
start(MANAGEDSERVER, "Server", ADMINSERVER_URL, block="true")
LOGFILE.writelines("Successfully Started " + MANAGEDSERVER + "\n")
print ''
print '============================================='
print '===> Successfully started ', MANAGEDSERVER, '  <==='
print '============================================='
print ''

E. stopManagedServers.py - Stops all the Managed Servers (comma separated list in properties file)

from java.io import FileInputStream
import java.lang
import os
import string
import datetime

logfile = "stopManagedServer.log"
LOGFILE = open(logfile,"a")
now = datetime.datetime.now()
LOGFILE.writelines("Stopping Managed Server - " + str(now) + "\n")

propInputStream = FileInputStream("weblogic.properties")
configProps = Properties()
configProps.load(propInputStream)

WEBLOGIC_USER = configProps.get("WEBLOGIC_USER")
WEBLOGIC_PASSWORD = configProps.get("WEBLOGIC_PASSWORD")
HOST = configProps.get("HOST")
NODEMANAGER_PORT = configProps.get("NODEMANAGER_PORT")
DOMAIN_NAME = configProps.get("DOMAIN_NAME")
DOMAIN_HOME = configProps.get("DOMAIN_HOME")
ADMINSERVER_NAME = configProps.get("ADMINSERVER_NAME")
ADMINSERVER_URL = configProps.get("ADMINSERVER_URL")
MANAGEDSERVERS = configProps.get("MANAGEDSERVERS")

while True:
try:
print 'Trying to connect to AdminServer...'
connect(WEBLOGIC_USER, WEBLOGIC_PASSWORD, ADMINSERVER_URL) 
break
except:
print 'Could not connect to Admin Server. Sleeping for 30 seconds...'
sleep(30)

LOGFILE.writelines("Connected to Admin Server Successfully...!!!" + "\n")
print ''
print '============================================='
print 'Connected to Admin Server Successfully...!!!'
print '============================================='
print ''

MANAGEDSERVERLIST = MANAGEDSERVERS.split(',')
for MANAGEDSERVER in MANAGEDSERVERLIST:
print 'Stopping managed server...'
print '###### MANAGEDSERVER = ', MANAGEDSERVER
shutdown(MANAGEDSERVER, "Server", ADMINSERVER_URL, force="true", block="true")
LOGFILE.writelines("Successfully stopped " + MANAGEDSERVER + "\n\n")
print ''
print '============================================='
print '===> Successfully stopped ', MANAGEDSERVER, '  <==='
print '============================================='
print ''

F. startup.bat - Batch file to start all servers sequentially

startAdminServer.bat && startManagedServer.bat

G. shutdown.bat - Batch file to stop all servers sequentially

stopManagedServer.bat & stopAdminServer.bat

5. Add scripts to system start-up / shutdown

Follow the steps provided here: http://technet.microsoft.com/en-us/library/cc770556.aspx to add startup.bat and shutdown.bat to the system startup and shutdown.

Thursday, December 22, 2011

SharePoint 2010 OAM Integration

This document outlines steps involved in Integrating SharePoint 2010 with OAM 10g.

Prerequisites:

  • SharePoint 2010 is installed and configured. You are able to create web applications and site collections.
  • OAM environment is installed and configured (Access Servers, Identity Servers etc).
  • Access Gate is configured for the webgate for webgate installation
  • A Policy Domain is configured for this resource (SharePoint 2010 URL).
  • Following Return Attributes are setup for Authorization Success in the Authorization Expression:

HeaderVar IMPERSONATE uid (or samaccountname)
HeaderVar SP_SSO_UID uid (or samaccountname)
COOKIE OAMAuthCookie uid (or samaccountname)
Assumptions

  • Web Application main site will be created using port 8081 (Claims based Authentication, NTLM only)
  • Main web application will be extended to Internet zone using port 80 and Forms based authentication only. Use CustomLDAPMembershipProvider and CustomRoleProvider for providers.


Step 1
In web.config of Central Admin (C:\inetpub\wwwroot\wss\VirtualDirectories\24195\web.config [on your server the folder name maybe different]) - Don't forget to make a backup first.

REPLACE
<roleManager><providers></providers></roleManager><membership><providers></providers></membership>
WITH
<roleManager><providers><addname="CustomRoleProvider" type="Microsoft.Office.Server.Security.LdapRoleProvider, Microsoft.Office.Server, Version=14.0.0.0, Culture=neutral, PublicKeyToken=71E9BCE111E9429C"server="[LDAP Server name]"port="389"useSSL="false"groupContainer=" dc=[Company],dc=com"groupNameAttribute="cn"groupMemberAttribute="uniquemember"userNameAttribute="uid"groupFilter="(ObjectClass=groupOfUniqueNames)"userFilter="(ObjectClass=inetorgperson)"scope="Subtree" /></providers></roleManager><membership><providers><addname="CustomLDAPMembershipProvider" type="Microsoft.Office.Server.Security.LDAPMembershipProvider, Microsoft.Office.Server, Version=14.0.0.0, Culture=neutral, PublicKeyToken=71E9BCE111E9429C"server="[LDAP Server Name]"port="389"useSSL="false"useDNAttribute="false"userNameAttribute="uid"userContainer="dc=[company],dc=com"userFilter="(objectClass=person)"scope="Subtree" otherRequiredUserAttributes="sn,givenname,cn" /></providers></membership>
Step 2
Deploy OAMCustomMembershipProvider.dll to GAC
(C:\Program Files (x86)\Microsoft SDKs\Windows\v7.0A\Bin\gacutil.exe - This could be different on your server)

gacutil -I D:\OAM\access\oblix\apps\webgate\OAMCustomMembershipProvider\OAMCustomMembershipProvider.dll

Alternatively drag and drop the OAMCustomMembershipProvider.dll to c:\windows\assembly

Step 3
In web.config of SecurityTokenServiceApplication Application (Application under SharePoint Web Services Site)
C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\14\WebServices\SecurityToken\web.config
Don't forget to make a backup first.

Add the below block right under </system.net>:
<system.web>
<membership>
<providers>
<add
name="CustomLDAPMembershipProvider" type="Oracle.CustomMembershipProvider, OAMCustomMembershipProvider,Version=1.0.0.0, Culture=neutral, PublicKeyToken=52e6b93f6f0427a1, processorArchitecture=AMD64"
server="[LDAP Server Name]"
port="389"
useSSL="false"
useDNAttribute="false"
userNameAttribute="uid"
userContainer="dc=[company],dc=com"
userFilter="(objectClass=person)"
scope="Subtree" otherRequiredUserAttributes="sn,givenname,cn" ValidationURL=http://[YourServerName]/ValidateCookie.html
DebugFile="D:\OAM\Logs\debug.log"
OAMAuthUser="OAMAuthCookie"/>
</providers>
</membership>
<roleManager enabled="true">
<providers>
<add
name="CustomRoleProvider" type="Microsoft.Office.Server.Security.LdapRoleProvider, Microsoft.Office.Server, Version=14.0.0.0, Culture=neutral, PublicKeyToken=71E9BCE111E9429C"
server="[LDAP Server Name]"
port="389"
useSSL="false"
groupContainer="cn=groups,dc=<Company>,dc=com"
groupNameAttribute="cn"
groupMemberAttribute="uniquemember"
userNameAttribute="uid"
groupFilter="(ObjectClass=groupOfUniqueNames)"
userFilter="(ObjectClass=inetorgperson)"
scope="Subtree" />
</providers>
</roleManager>
</system.web>

Make sure the validation URL works (http://[YourServerName]/ValidateCookie.html) on your server. Put a ValidationCookie.html file in the root of "80" site with anything in it (Hello is ok). Make sure D:\OAM\Logs folder exists.

If there are issues with OAM integration this line under <behavior name="SecurityTokenServiceBehavior"> would help with debugging:
<serviceDebug includeExceptionDetailInFaults="true"/>

Step 4
In the web.config of "80" site (D:\inetpub\wwwroot\wss\VirtualDirectories\80) - Don't forget to make a backup first.

Add the following lines under <providers> for <membership> node

<add name="CustomLDAPMembershipProvider"
type="Microsoft.Office.Server.Security.LDAPMembershipProvider, Microsoft.Office.Server, Version=14.0.0.0, Culture=neutral, PublicKeyToken=71E9BCE111E9429C"
server="[LDAP Server Name]"
port="389" useSSL="false" useDNAttribute="false" userNameAttribute="uid"
userContainer="dc=[company],dc=com" userFilter="(objectClass=person)"
scope="Subtree" otherRequiredUserAttributes="sn,givenname,cn" />

Add the following lines under <providers> for <roleManager> node
<add name="CustomRoleProvider"
type="Microsoft.Office.Server.Security.LdapRoleProvider, Microsoft.Office.Server, Version=14.0.0.0, Culture=neutral, PublicKeyToken=71E9BCE111E9429C"
server="[LDAP Server Name]" port="389" useSSL="false" groupContainer=" dc=[company],dc=com"
groupNameAttribute="cn" groupMemberAttribute="uniquemember"
userNameAttribute="uid" groupFilter="(ObjectClass=groupOfUniqueNames)"
userFilter="(ObjectClass=inetorgperson)" scope="Subtree" />

Step 5
In the Default.aspx of the " 80" site (D:\inetpub\wwwroot\wss\VirtualDirectories\80\_forms\Default.aspx) - Don't forget to make a backup first.

Add the following lines just above </asp:Content>
<asp:HiddenField EnableViewState="false" ID="loginTracker" runat="server" value="autoLogin" />
<%bool autoLogin = loginTracker.Value == "autoLogin";%>
<script runat="server">
void Page_Load()
{
signInControl.LoginError += new EventHandler(OnLoginError);
NameValueCollection headers = Request.ServerVariables;
NameValueCollection queryString = Request.QueryString;
string loginasanotheruser = queryString.Get("loginasanotheruser");
string username = Request.ServerVariables.Get("HTTP_SP_SSO_UID");
HttpCookie ObSSOCookie = Request.Cookies["ObSSOCookie"];
bool isOAMCredsPresent = username != null && username.Length > 0 && ObSSOCookie != null && ObSSOCookie.Value != null;
bool signInAsDifferentUser = loginasanotheruser != null && loginasanotheruser.Contains("true");


if (isOAMCredsPresent)
{
//Handling For UTF-8 Encoding in HeaderName
if (username.StartsWith("=?UTF-8?B?") && username.EndsWith("?="))
{
username = username.Substring("=?UTF-8?B?".Length, username.Length - 12);
byte[] decodedBytes = Convert.FromBase64String(username);
username = Encoding.UTF8.GetString(decodedBytes);
}
}
if (isOAMCredsPresent && loginTracker.Value == "autoLogin" && !signInAsDifferentUser)
{
bool status=Microsoft.SharePoint.IdentityModel.SPClaimsUtility.AuthenticateFormsUser (new Uri(SPContext.Current.Site.Url),username,"ObSSOCookie:"+ObSSOCookie.Value);
if(status){
if (Context.Request.QueryString.Keys.Count > 1)
{
Response.Redirect(Context.Request.QueryString["Source"].ToString());
}
else
{
Response.Redirect(Context.Request.QueryString["ReturnUrl"].ToString());
}
}
else{
loginTracker.Value = "";
}
}
else
{
// DO NOTHING
}
}
void OnLoginError(object sender, EventArgs e)
{
loginTracker.Value = "";
}
</script>

Step 6
To disable Persistent cookie run the following powershell script:
Get-PSSnapin -RegisteredAdd-PSSnapin Microsoft.SharePoint.Powershell$sts = Get-SPSecurityTokenServiceConfig$sts.UseSessionCookies = $true$sts.Update()iisreset
Step 7
Go App management in Central admin. Click on the main web app (8081) and click on User Policy, Add yourself (from the FBA account) with Full Control.

Step 8
Add the webgate to "80" site in IIS Manager.

Step 9
Test the site by going to http://[yourserver]. You should be able to login as yourself and you should have full access. Give other LDAP users access in the next step.

Step 10
Go to http://[yourserver]/_layouts/user.aspx. Click on "Site Visitors", click on new, search for All and add "All Authenticated Users" and click Ok.

Now all users who can authenticate against OAM have read access to the site.

Wednesday, May 12, 2010

Setting up One Way Trust for SharePoint installations

We just installed our SharePoint servers in the perimeter network (DMZ). Since our domain controllers are in the corporate network we had to configure a separate domain in the DMZ and setup one way trust between the 2 domains (DMZ domain trusts the internal domain). We had to struggle a bit becuase of the firewall rules that needed to be created and changes that need to be made to the domain controllers.

Firewall Ports to be opened

Microsoft-DS traffic: tcp-445, udp-445
LDAP: tcp-389 (or tcp636 if using SSL)
LDAP Ping: udp-389
Kerberos authentication protocol: tcp-88, udp-88
DNS: tcp-53, udp-53
Net Logon Service: tcp-135, udp-135
Dynamic RPC ports: 5000-5100

Note that these ports need to be opened not only for the DCs in DMZ to communicate with the internal DCs but also any server that needs to authenticate against the internal DCs which in our case were the SharePoint web front ends.

Limit the Dynamic RPC ports to 5000-5050

Net Logon service uses RPC endpoint mappers (tcp and udp 135) for initial handshake and use high ports for subsequent communucation. Since it is not secure to open all the high ports we need to limit the range of dynamic RPC ports(5000-5100 in our case). To limit the range set the following registry key:
[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Rpc\Internet]

"Ports"=REG_MULTI_SZ:5000-5100

Note that this limiting of dynamic ports mainly needs to be done on all the DCs in the internal network. We went ahead and did this on the DMZ DCs as well.

OAM Webgate configuration for ARR Reverse Proxy

I just configured a reverse proxy for our SharePoint portal site using IIS7 and ARR. I installed webgate with Windows Authentication (IWA) turned on since we wanted employees to be single signed on. However I noticed significant performance loss going through this reverse proxy as opposed to logging in directly to the SharePoint server.

I set the CachePragmaHeader and CacheControlHeader to public instead of the default no-cache to improve performance significantly. I obviously also removed the custom 401 error message which send a whole HTML file. See blog below.

By the way, if people are unable to download word documents or excel spreadsheets from your websites that is protected by webgate update the webgate configuration and set CachePragmaHeader and CacheControlHeader to public. That should resolve the issue (in most cases).

Wednesday, May 21, 2008

IWA Performance consideration

IWA (Integrated Windows Authentication) is a cool thing where users need to authenticate again if their computer is already authenticated to the Windows network. However if you have users in remote locations they might experience significant performance degradation with IWA turned on. The reason for this is IWA requests the browser to send its NTLM authentication key. There will be 2 401 (Unauthorized) responses before the 200 (OK) response.

This article by Matthew Langston explains how IWA works: http://confluence.slac.stanford.edu/display/Gino/Integrated+Windows+Authentication?decorator=printable. So I don't need to explain it again here.

Since you can't eliminate the 2 401 errors (That's the way IWA works), you can improve on performance by cutting down the amount of data transfered over the network. By default IIS is configured to send the standard HTML file when a 401 error occurs. You need to change the setting in the "Custom Errors" tab (in IIS website properties) and set the 401 errors to send default text message instead of the HTML file. This will reduce amount of data transferred from around 2KB to about 83 bytes. You will see substantial improvement in performance.

This article explains how to change the "custom Errors" properties (Your website setting maybe slightly different): http://support.microsoft.com/kb/817322/en-us

Peace!

Thursday, May 8, 2008

Coding Standards and Guidelines

I was asked to create a coding standards document where I work. Oh yeah, I did not do it because I love to do this kind of work. So I outsourced by job to my best friend Google ;-) I got several hits. I compiled a coding standards and guidelines document with the help my buddy. I would like to give it back to from where I got. So here it is:

Coding Standards



1. Introduction


Application developers need to adhere to certain coding standards in order to
enhance the readability and maintainability of the application.

Well written software offers many advantages. It will contain fewer bugs and
will run more efficiently than poorly written programs. Since software has a
life cycle and much of which revolves around maintenance, it will be easier for
the original developer(s) and future keepers of the code to maintain and modify
the software as needed. This will lead to increased productivity of the
developer(s). The overall cost of the software is greatly reduced when the code
is developed and maintained according to software standards.

This guide makes a distinction between standards and guidelines. Standards are
rules which programmers are expected to follow. Guidelines can be viewed as
suggestions which can help programmers write better software and are optional,
but highly recommended.


2. Internal Documentation Standards


If done correctly, internal documentation improves the readability of a software
module. A file containing one of more software modules should have comment block
at its beginning containing the following basic information:



  • The name of the author who created the file
  • The date the file was created
  • The author’s development group
  • Description (overview of the purpose of the modules)

Note that a module is a method, function, or subroutine.

Each module contained within the source file should be preceded by a block of comments showing the following:



  • The name of the module
  • The name of the original author (if the module author is different than the author of the file.)
  • The date the module was created
  • A description of what the module does
  • A list of the calling arguments, their types, and brief explanations of what they do
  • A list of required files and/or database tables needed by the routine, indicating if the routine expects
    the database or files to be already opened
  • Return values
  • Error codes/exceptions


3. Coding Standards


3.1 Indentation:

Proper and consistent indentation is important in producing easy to read and
maintainable programs. Indentation should be used to:


  • Emphasize the body of a control statement such as a loop or a select statement
  • Emphasize the body of a conditional statement
  • Emphasize a new scope block


Tabs should be used for indentation.



Examples:

/* Indentation used in a loop construct. */

for (int i = 0; i < number_of_employees; ++i)

{

   
total_wages += employee[i].wages;

   
if (total_wages > 1,000,000)

   
{

       
System.out.println(“Over one million”);

   
}

}



// Indentation used in the body of a method.

package void get_vehicle_info ( )

{

   
System.out.println ( “VIN: “ + vin ) ;

   
System.out.println ( “Make: “ + make ) ;

   
System.out.println ( “Model: “ + model ) ;

   
System.out.println ( “Year: “ + year ) ;

}




3.2 Inline Comments

Inline comments explaining the functioning of the subroutine or key aspects of
the algorithm shall be frequently used.



Inline comments should be used to make the code clearer to a programmer trying
to read and understand it. Writing a well structured program lends much to its
readability even without inline comments. The bottom line is to use inline
comments where they are needed to explain complicated program behavior or
requirements. Use inline comments to generalize what a block of code,
conditional structure, or control structure is doing. Do not use overuse inline
comments to explain program details which are readily obvious to an
intermediately skilled programmer.



3.3 Structured Programming

Structured (or modular) programming techniques shall be used. GO TO statements
shall not be used as they lead to “spaghetti” code, which is hard to read and
maintain



3.4 Classes, Subroutines, Functions, and Methods

Keep subroutines, functions, and methods reasonably sized. This depends upon the
language being used. A good rule of thumb for module length is to constrain each
module to one function or action (i.e. each module should only do one “thing”).
If a module grows too large, it is usually because the programmer is trying to
accomplish too many actions at one time.



The names of the classes, subroutines, functions, and methods shall have verbs
in them. That is the names shall specify an action, e.g. “getName”, “computeStatistics”.



3.5 Source Files

The name of the source file or script shall represent its function. All of the
routines in a file shall have a common purpose.



3.6 Variable Names

Variable shall have mnemonic or meaningful names that convey to a casual
observer, the intent of its use. Variables shall be initialized prior to its
first use. Variable names shall be defined in Pascal Case (e.g. getUserInfo,
setMemberProfile).



3.7 Use of Braces

In some languages, braces are used to delimit the bodies of conditional
statements, control constructs, and blocks of scope. Programmers shall use the
following bracing style:



for (int i = 0 ; i < 100; i++)

{

   
/* Some work is done here. */

}




Braces shall be used even when there is only one statement in the control block.
For example:

Bad:

if (i <= 0)

printf (“Positive number required.\n”);



Better:

if (i <= 0)

{

   
printf (“Positive number required.\n”);

}





4. Coding Guidelines



General coding guidelines provide the programmer with a set of best practices
which can be used to make programs easier to read and maintain.

4.1 Line Length

It is considered good practice to keep the lengths of source code lines at or
below 80 characters. Lines longer than this may not be displayed properly on
some terminals and tools.

4.2 Spacing
The proper use of spaces within a line of code can enhance readability. Good
rules of thumb are as follows:

  • A keyword followed by a parenthesis should be separated by a space.
  • A blank space should appear after each comma in an argument list.
  • All binary operators except “.” should be separated from their operands by spaces. Blank spaces should never separate unary operators such as unary minus,
    increment (“++”), and decrement (“—“) from their operands.
  • Casts should be made followed by a blank space.



Example:
Bad:
cost=price+(price*sales_tax);
Better:
cost = price + ( price * sales_tax );



4.3 Wrapping Lines
When an expression will not fit on a single line, break it according to these
following principles:

  • Break after a comma

    Example:

    fprintf ( stdout , “\nThere are %d reasons to use standards\n” ,
    num_reasons ) ;

  • Break after an operator

    Example:

    long int total_apples = num_my_apples + num_his_apples +
    num_her_apples ;

  • Prefer higher-level breaks to lower-level breaks

    Example:

    Bad:

    longName1 = longName2 * (longName3 + LongName4 –
    longName5) + 4 * longName6 ;

    Better:
    longName1 = longName2 * (longName3 + LongName4 – LongName5)
    + 4 * longName6 ;

  • Align the new line with the beginning of the expression at the same level on the previous line.

    Example:
    total_windows = number_attic_windows + number_second_floor_windows +
    number_first_floor_windows ;




4.4 Program Statements
Program statements should be limited to one per line. Also, nested statements
should be avoided when possible.
Examples:
Bad:
number_of_names = names.length ; b = new JButton [ number_of_names ] ;


Better:
number_of_names = names.length ;
b = new JButton [ number_of_names ] ;


Bad:
strncpy ( city_name , string , strlen ( string ) ) ;

Better:
length = strlen ( string ) ;
strncpy ( city_name , string , length ) ;



4.5 Use of Parentheses
It is better to use parentheses liberally. Even in cases where operator
precedence unambiguously dictates the order of evaluation of an expression,
often it is beneficial from a readability point of view to include parentheses
anyway.
Example:

Acceptable:
total = 3 – 4 * 3 ;

Better:
total = 3 – ( 4 * 3 ) ;



4.6 Coding for Efficiency vs. Coding for Readability
There are many aspects to programming. These include writing software that runs
efficiently and writing software that is easy to maintain. These two goals often
collide with each other. Creating code that runs as efficiently as possible
often means writing code that uses tricky logic and complex algorithms, code
that can be hard to follow and maintain even with ample inline comments.

The programmer needs to carefully weigh efficiency gains versus program
complexity and readability. If a more complicated algorithm offers only small
gains in the speed of a program, the programmer should consider using a simpler
algorithm. Although slower, the simpler algorithm will be easier for other
programmers to understand.

4.7 Meaningful Error Messages
Error handling is an important aspect of computer programming. This not only
includes adding the necessary logic to test for and handle errors but also
involves making error messages meaningful. Error messages should also be stored
in way that makes them easy to review.


 





Secure Coding Guidelines


Protecting access to your data source is one of the most important goals when
working on the security of your application. To help limit access to your data
source it is imperative to keep connection information such as Username,
Password, data source name, etc. private.


1. Read Only DB User


Applications code must connect to the database as a database user who has read
only access to the data and can’t update or change any data or objects. Only
where necessary (such as admin pages that require update or insert) connect as
another user who has limited privileges to write to only the required database
objects. Also strongly consider using Windows Integrated Authentication to
access SQL Server in your code. If it can’t be used for any reason, avoid
storing clear text password instead store the password encrypted.



2. Validating Inputs


Before using the HTTP inputs provided by end users in a database query, validate
them using regular expressions or other means. For example, the following
ensures that a userid value is an 8-character alphanumeric string.



[Visual Basic] Copy Code

Public Static Function ValidateUserid(inString As String) As Boolean

   
Dim r As Regex = New Regex("^[A-Za-z0-9]{8}$")

   
Return r.IsMatch(inString)

End Function



[C#]

public static bool ValidateUserid(string inString)

{

   
Regex r = new Regex("^[A-Za-z0-9]{8}$");

   
return r.IsMatch(inString)

}




Validating whether the input data is per our expectation reduces the risk of SQL
injection attacks substantially.




3. Using Parameters (Prepared Statements)


Parameters provide a convenient method for organizing values passed with a SQL
statement or to a stored procedure. Additionally, parameters can guard against a
SQL Insertion attack by ensuring that values received from an external source
are passed as values only, and not part of the SQL statement. As a result, SQL
commands inserted into a value are not executed at the data source. Rather, the
values passed are treated as a parameter value only. The following code shows an
example of using a parameter to pass a value.



[Visual Basic] Copy Code

'Retrieve CustomerID to search for from external source.

Dim custID As String = GetCustomerID()



Dim selectString As String = "SELECT * FROM Customers

                              
WHERE CustomerID = @CustomerID"



Dim cmd As SqlCommand = New SqlCommand(selectString, conn)

cmd.Parameters.Add("@CustomerID", SqlDbType.VarChar, 5).Value = custID



conn.Open()

Dim myReader As SqlDataReader = cmd.ExecuteReader()

'Process results.

myReader.Close()

conn.Close()



[C#]

// Retrieve CustomerID to search for from external source.

string custID = GetCustomerID();



string selectString = @"SELECT * FROM Customers

                        
WHERE CustomerID = @CustomerID";



SqlCommand cmd = new SqlCommand(selectString, conn);

cmd.Parameters.Add("@CustomerID", SqlDbType.VarChar, 5).Value = custID;



conn.Open();

SqlDataReader myReader = cmd.ExecuteReader();

'Process results.

myReader.Close();

conn.Close();


4. Keep Exception / Server Error Information Private


Attackers often use information from an exception, such as the name of your
server, database, or table to mount a specific attack on your system. Because
exceptions can contain specific information about your application or data
source, you can help your application and data source better protected by only
exposing information to the client that is required.



Use friendly error messages such as “Database Connection failed. Please contact
the Customer Support at support@company.com”. Do not display the server errors
or the Exception stack to the end user in production environments.