WebSphere - Node synchronization status shows as question mark

In this case you need to manually synchronize the node with Deployment Manager/NDM.

Go to node bin directory and run the following command. Please make sure nodeagent and node is stopped.

syncNode.sh 'deployment Manager Host Name' 'soap Connector Port-8879' -username wasadmin -password 'password'

Start nodeagent and see if the synchronization status is green. Then start the node.

Reconfigure WebSEAL after the crash

You might run into a situation where the server on which WebSEAL instance is configured just crashes or gets rebuilt without an opportunity to unconfigure the WebSEAL instance.

WebSEAL reconfiguration will fail with the following error:

HPDMG0759W The user name already exists in the registry.

You need to delete the user from TAM registry with user name "default-webseald-" before you can reconfigure WebSEAL instance.

WebSEAL SSO Configuration with SAP Portal

Steps involved in configuration of SSO between SAP Portal and WebSEAL:

1. Install AMWeb ADK and AMWeb RTE on WebSEAL
2. Download platform specific sapseculib and sapssoext libraries from SAP
3. Download verify.pse from the SAP Portal environment and copy to the WebSEAL box
4. Update WebSEAL configuration file with necessary properties:

Please refer to this link for more information

http://www.ibm.com/developerworks/tivoli/library/t-authsaptam/index.html

Considerations:

1. SAP has stopped supporting 32-bit libraries, so configuring SSO on WebSEAL 32-bit is at your own risk. The solution may work but you will not get support from SAP :)

2. Code that IBM has provided has a method implementation missing: getSSOLibrary. You will need to implement this method on your own.

Happy SSOing :)

TAM on Suse Linux

Tivoli Access Manager works on Suse Linux 32-bit operating system. On 64-bit operating system, all components except WebSEAL are supported. Though you can install WebSEAL on 64-bit, all the libraries that it uses internally are 32-bit.

Policy Server reconfiguration in the event of Directory Server crash

In the event of ITDS crash, Policy Server can not be unconfigured using pdconfig. Use /opt/PolicyDirector/sbin/PDMgr_unconfig to unconfigure Policy Server. Authorization server can be reconfigured once Policy server is up. Please refer to my post below on reconfiguring WebSEAL in such event.

Components such as Web Portal Manager and others need not be unconfigured. Check the username of the user account used during initial configuration. Create the user with same user credentials on Policy Server. Web Portal Manager should work as expected after this.

All About Common Auditing and Reporting Service

Common Auditing and Reporting Service(CARS) is capable of collecting events from Master Policy Server and WebSEAL and can process the data so that reports can be viewed through Tivoli Common reporting package or any other third party reporting products.

CARS can be deployed in clustered and non clustered environments. Installation and configuration of CARS is not straight forward and there are few steps that need to be verified/corrected post CARS configuration to ensure it functions as expected.

CARS is better known to work on AIX because its an IBM component and they probably unit test their code on AIX :)

Here are few things that we need to verify post CARS Configuration:

1. Ensure stored procedures run properly. From DB2 command line, execute the stored procedures to make sure they are running properly
2. Run Test Connection on eventxml data source. In a clustered environment, its important that the data source connection is successful on both nodes.
3. Make sure DB2 client is installed on both the nodes in a clustered environment
4. If MPS/WebSEAL tries to talk to CARS over SSL:

• Ensure that CARS application roles are mapped to the user id that MPS and WebSEAL use to communicate to CARS.
• After running amauditcfg on MPS and WebSEAL, verify if clientPassword is set in the config file( clientUsername, clientPassword, key database path, stash file path are mandatory if MPS/WebSEAL tries to talk to CARS over SSL).
• Ensure application security is enabled on the deployment manager profile where CARS cluster resides.

Happy auditing :)

Session Management Server on WebSphere cluster

When installing and configuring Session Management Server(SMS) installation on WebSphere cluster, make sure you consider the following things:
1. Make sure to upgrade the objectgrid package which comes with WebSphere to the supported version. Please refer to Tivoli Access Manager release notes for the objectgrid version and fixpack details
2. On the WebSphere servers, If there are WebSphere profiles other than cluster member nodes, make sure you stop all of them. Only the WebSphere Dmgr profile and its associated cluster member nodes should be running.
3. Even after installation and configuration, if you start the profiles other than WebSphere Dmgr and its member nodes first, then SMS console and its associated components will thrown the following exception and will fail to work:
"com.tivoli.am.sms.DSessException: The Session Management Server has encountered an error and was unable to complete the operation"

Configure Policy Server failover in non-AIX operating systems

Policy Server failover is achieved using HACMP on AIX environment. How do you configure failover in operating systems other than AIX such as Windows.

One of the ways that I could see is:

1. Take nortan ghost of MPS machine and on a new machine with the same OS configuration, extract the ghost.

2. Keep the second machine disconnected from network. This machine will have same hostname as the primary MPS machine.

3. On a periodic basis, take MPS backup from primary using pdbackup command and restore the backup on second machine(call it backup machine).

4. In the event of primary MPS failure, disconnect primary from network and connect backup MPS machine to network.

Why we chose TAM over SAM ?

There are lot of competitive advantages of TAM over SAM and vice versa. I dont want to discuss all that at the moment :)

Sun Access Manager (SAM) installable is not available on AIX platform. You know the reason. Dont you ? :) Okay, here it is:

IBM has a proprietary JRE for AIX platform. Sun Access Manager works only with Sun's JRE not any other.

And if your production environment is AIX, you dont have an option but to choose TAM over SAM.

What is Authorization Server ?

Authorization Server is a client side replica of Policy Server. It needs to be installed with Access Manager Java Runtime environment. It caches policy server data and synchronizes it on a regular basis. Even though Policy Server goes down, Authorization server still serves your requests.

However there is a limitation on the data that authorization server can provide. Authorization server API provides ways of accessing basic attributes of user, group and ACL. PDPrincipal object is one such example API. For using Authorization API, developers have to first create
PDAuthorizationContext and supply that as input to any authorization API. If you want to create user or group, you need to go through Policy Server API.

With TAM 6.0, Authorization and Policy Server API are clubbed into one and are called Access Manager Application Development Kit(AMADK). com.tivoli.pd.jazn API are specific to Authorization server in AMADK.

Tivoli Policy Server data backup/restore

To backup/restore Tivoli configuration

This is to backup/restore TAM configuration data which is not stored in LDAP

pdbackup –action backup

–list /opt/PolicyDirector/etc/pdinfo.lst

–path /data/tivoli/backup

–file pdbackup

Once the above command is executed, we can find an archive file by name pdbackup.tar under the folder /data/tivoli/backup.

To restore use the same command with -action restore option.

To backup/restore Tivoli Policy Server configuration

This is to take backup of tivoli policy server configuration. We can either user idsdb2ldif or idsdbback.

Approach I
a) Use command "idsdb2ldif" to take ldif backup - It takes backup of ldap schema files & backup ibmslapd.conf files.
b) Use command "idsldif2db" for data restore.

Approach II
"idsdbback" for backup and "idsdbrestore" to restore. For this to work, the software versions and setup on both machines should be same.

1. If data size is not much, use "idsdb2ldif" and "idsldif2db" approach.
2. It is suggested to use both approaches (idsdb2ldif or idsdbback )

My two cents on Tivoli Directory Server Clustering

Directory server clustering

It is advised that directory server and DB2 should reside on the same machine to eliminate network latency. Directory server is optimized to work with DB2 so it’s advised that Directory server be configured with DB2 not any other databases like Oracle.

Master-Slave configuration

For reliability, both master and slave should ideally reside on different servers using their own database instances. This configuration is recommended for high availability in addition to reliability.

Without Load Balancer

In a master-slave environment without a load balancer, all requests are routed to the master directory server. Policy server can be made aware of the Slave LDAP Server by putting the configuration (Slave hostname and port) in the ldap.conf file available in /etc. Once replication is setup, there is a default periodic interval for scheduled replication. There is an option to override and do a manual force replication. Slave can be upgraded as Master in event of failure. This is a manual process and no automated scripts are available as a part of the product.

TAM can differentiate between read and write updates to LDAP. One can separately configure a set of replicas for read type of operations and another set for read/write operations. Also, priority can be set for the LDAP servers in the ldap.conf file.

With Load balancer (Websphere Edge Server)

In a master-slave environment without a load balancer, all write operations are channeled to master directory server. Reads are load balanced between master and slave. A scheduled replication process takes care of replicating the master data into the slave database. In event of failure of the master directory server, the load balancer channels all reads to the slave, writes are not allowed on the slave and would fail eventually. In case of master being down, slave has to be manually configured as master (takes about 10-15 minutes of downtime) to allow writes. Failover for read is automated in this case. To avoid write operations getting lost, we need to implement some kind of queuing mechanism to preserve the write operations. There is a lag between the write to the master and the replication from master to slave. As a result it is quite possible that a write immediately followed by a read might not see the updated data.

Replication can be scheduled through TDS Web Admin tool / command line utility. Policy Server can be made aware of the Slave through the ldap.conf file. Slave can be upgraded as Master if so needed.

Questions

How to configure policy server for two directory server instances in case Directory server is clustered?
Answer: The ldap master & slave replica hostnames and ports need to be mentioned in the ldap.conf file. The replicas can be configured as read-only or read-write.

In a Master-Slave directory server cluster scenario, if master goes down and if manually bring up slave as Master, will Policy Server pick fall back on this automatically?
Answer: In this scenario we can configure the both the ldap replicas in the Policy Server ldap.conf file as read-write with a priority. The Master has to have a much higher priority than the slave so that read-write requests will almost always go to the LDAP master. In event of a failure, the policy server will then talk to the slave. The catch would be to ensure that the slave promotes to master before Policy server talks to slave. Thus, recommended setup is a Master-Master setup if timing is very critical.

What is the utility which will replicate data between master and slave?
Answer: There is no separate utility to replicate master and slave. This replication is available through Tivoli Directory Server Web Administration tool.

Master-Master configuration

Both Masters should ideally reside on different servers using their own databases.
For High Availability, it is recommended that both the masters reside on separate physical machines. This configuration thus needs separate database instances for both the masters.

Without Load Balancer
- Not Applicable -

With Load balancer (Websphere Edge Server)
All reads/writes are load balanced between the Master servers. A scheduled sync up process replicates data to and from the masters (increasing network usage). Policy server is configured with two master servers. It is highly likely that user might not see the data just inserted/updated.
Since, both replicas are masters, the writes can be handled by both the replicas. But it is a good practice to ensure that writes still go to one of the masters and only in the event of a failure, it fails over to the other master. This is to ensure lesser network bandwidth & IO. Load balancer will be configured to have higher priority to one of the masters. The other master just ‘listens’ to replication till it becomes a master.

Hence, there needs to be two-way replication so as to keep both the masters in-sync. Policy server can be made aware of the two masters by adding appropriate parameters in the ldap.conf file.

Policy server clustering

Policy server can be clustered only using OS clustering (HACMP option in IBM AIX servers). The Policy server clustering feature is only supported on AIX (via HACMP).

Solution to AJAX problem with WebSEAL

To make AJAX calls work properly, WebSEAL junctions needs to be created with scripting support.

PDAdmin Command prompt - Create the WebSEAL junction with scripting support ie using the -j option

Web Portal Manager - check on "Enable scripting support" under scripting support tab in junction creation screen.

Configure Tivoli Access Manager for multiple suffixes

During configuration of tivoli we create a suffix and tivoli access manager by default associates the following three ACLs to the suffix:

1.cn=securitygroup,secauthority=default
2.cn=ivacld-servers,cn=securitygroups,secauthority=default
3.cn=remote-acl-users,cn=securitygroups,secauthority=default

When we create another suffix and want tivoli access manger to recognize it, we need to add these ACLs manually through Directory Server Admin Console.

Login in to Directory Server Admin Console - Goto Directory Management - Click on Manage Entries - Select the suffix that is created - Expand the 'Select Action' dropdown and select Edit ACL -Click on Non-Filtered ACLs - add the above ACLs.

HPDMG0769E error

HPDMG0769E There were insufficient LDAP access privileges to allow Tivoli Access Manager to create and delete entries in the registry.

Cause: This error may happen if Policy Server configuration in LDAP is disturbed.

Solution: Unconfigure and configure Policy Server using pdconfig.

What if Policy Server crashes ?

In development environment, Policy server instance is centrally located. JRTE,ADK and WebSEAL instances are installed and configured on development machines to point to Policy Server.

When Policy Server crashes for whatsoever reasons, WebSEAL, AMADK instances can not be unconfigured and configured to a new Policy Server instance. It throws a message saying "policy server instance not available". we can not uninstall WebSEAL or AMADK with out unconfiguration. In such cases, following steps will help you to configure WebSEAL and AMADK instances.

1. Take pd.sth and pd.kdb from /var/PolicyDirector/keytab folder (in Linux) or equivalent as per Operating System.
2. Copy them to /Tivoli/PolicyDirector/keytab on development machines. If these files are already there, overwrite them with files which you copied from central Policy Server
3. Open ldap.conf and pd.conf on development machine at /Tivoli/PolicyDirector/etc folder. Replace old policy server machine name with new policy server machine name.
4. Go to Tivoli Access Manager - configuration window. Click on unconfigure WebSEAL. It will prompt for sec_master password after which it should unconfigure successfully.
5. Repeat Step 4 for AMADK as well.
6. Now you can configure AMADK and WebSEAL to latest Policy Server instance.

Web Portal Manager 6 - Object space Copy/Paste

Copy/Paste of object space doesnt work in Tivoli Access Manager Web Portal Manager 6.0. You need to download and install Fix pack 4 to make it work.

Specify computer name during Tivoli installation, not IP

During the installation of IBM Tivoli Access Manager's components such as JRTE, AMADK and WebSEAL , please specify Tivoli Policy Server machine name not IP. Please make sure you put an entry against that name in your hosts file under WINDOWS\system32\drivers\etc.

If you dont do this, policy server API calls will take considerably more time than usual.

Problem with DB2 used by Directory Server

If you install DB2 as the Database for IBM Directory server, make sure you change DB2 password before you restart it for the first time. Otherwise, Directory server fails to start and there is no way to figure out why its not starting.

WebSEAL session gets reset when used with AJAX

When you configure an application on Websphere application server 6.1 to a WebSEAL junction, requests to all resources inside the application go through WebSEAL via WebSEAL Junction. When user is authenticated and when XMLHTTPRequest object is used for asynchronous request/response in a JSP, HTTP session (JSESSIONID from WAS 6.1) is reset. As a result of that, custom session objects are lost. It looks like WebSEAL session id (PD-H-SESSION-ID) also is getting reset. Eventually, user is thrown back to login page for re-authentication.

WebSEAL installation issue on Windows XP

WebSEAL installation fails on Windows XP. It gives the following error:

"The WebSEAL instance 'default' failed to configure".

Not to panic. Just do that following.


1. Open webseald-default.conf in the C:\Program Files\Tivoli\PDWeb\etc subdirectory.
2. Locate the [webseal-config] stanza, and change the value of the "status" parameter from "partial" to "config"
3. Click "Refresh" in the Access Manager WebSEAL Configuration window. The status for the default instance should now show as "Stopped".
4. Open the Services application from the Start / Control Panel / Administrative Tools menu.
Update the Log on properties of the "Access Manager WebSEAL-default" service, so that the service runs under the local Administrator account.

Once you do this, You will be able to start the "Access Manager WebSEAL-default" service successfully. For more details:

http://www-128.ibm.com/developerworks/tivoli/library/t-tamxp/index.html