Prerequisites For My cf.Objective() Presentation on Tomcat

Quick note to anyone planning to attend my “Running Multiple CFML Engines on Apache Tomcat” talk at cf.Objective() — even though this is only a one-hour session, with just a bit of prep work you can easily turn this into a hands-on session since I only have a few slides and it will be mostly demo. You don’t have to follow along to get a ton of great info from this session, but if you want to follow along please grab the following ahead of time:

• Java (I assume this is already on most attendee’s machine and configured properly)
• Apache Tomcat
• Apache Web Server
• Dealer’s choice/mix and match: Open BlueDragon, Railo, and Adobe ColdFusion

Some additional notes:

• You do NOT need to install Tomcat ahead of time
• You SHOULD install Apache ahead of time
• If you want to use Adobe CF as one of your engines, you’ll want to run the installer ahead of time and for the installation type choose “generate a WAR file” and have that available on your laptop. Note that even if you have Adobe CF installed on your machine already, you can run the installer again and generate a WAR file without affecting your existing installation.
• For Open BlueDragon and Railo, grab the WAR files and have those handy
• Your operating system doesn’t matter–all the Tomcat stuff is pure Java, so whether you’re on GNU/Linux, Windows, or Mac it’s all good.

If you have questions/concerns ahead of time please comment here or email me. See you at cf.Objective()!

CFML and Oracle Stored Procedures: Experimentation and Solutions

I love a challenge. Most of the time when someone tells me something can't be done, I have to prove it to myself even if these attempts are frequently exercises in futility. And even if said something actually cannot be done, I always feel like I learn a ton during the process.

Such was the case when I was working with a coworker recently on getting data back from some Oracle stored procedures into either Open BlueDragon or Railo. Before I proceed, let me state very clearly that I hardly ever touch Oracle so if I'm off-base on any of this I'm happy for an Oracle expert to educate me. In my defense, I will say I did quite a bit of searching on this topic, and more hours of experimentation than I care to admit, so I wasn't just screaming "Oracle sucks!" and not actually trying to intelligently and logically get things to work. Also bear in mind that in this situation we didn't write the storedprocs and we do not have the ability to alter them. (Necessity is the mother of invention and all that.)

Back to the task at hand. The storedprocs in question return Oracle REF CURSORs, which is basically Oracle's way of thumbing their ever-expanding nose at the world by doing things differently than everyone else simply because they want to. (Oracle fans, if there's a legitimate explanation for this utter nonsense I'm all ears, because I certainly couldn't find one.)

If your first thought is, "But this just works on Adobe ColdFusion!" you are in fact correct. This is because Adobe CF ships with DataDirect drivers that handle getting an Oracle REF CURSOR into a format that can be used by CF. Because REF CURSORs ain't your plain old Java ResultSets like the entire rest of the known database universe uses, my assumption is that there's some translation that goes on either in the driver, or on the CF side, or both, to make this work. But in other CFML engines if you're using the plain old JDBC drivers–yes, even Oracle's own JDBC drivers–this doesn't "just work."

Let's assume it's entirely in the driver. I could either purchase the DataDirect drivers (expensive, and silly to have to do this for this one seemingly small thing), or I could use the ones that ship with CF. Well, a little birdie told me (ahem) that if you try to use the ones that ship with CF anywhere but with CF you'll get a licensing error when trying to run queries. So that solution's out.

You might also think you could simply call the storedproc from within a normal CFQUERY tag as opposed to using CFSTOREDPROC and an out parameter. Good thought, but even if you get the syntax figured out, since the storedprocs in question are looking for an out parameter to be passed into the storedproc (figure that one out), there's no real way to get the data returned from the storedproc into a variable within the SQL statement that you can actually access. I spent a lot of time on this and got the storedproc itself executing fine, but getting at the data was where I spent the bulk of my experimentation time, and I finally gave up.

So where does that leave us? Long and short of it is that without using the DataDirect drivers (even if that would in fact work), there is no way that I could come up with that would allow calling an Oracle storedproc that returns a REF CURSOR from OpenBD or Railo that would work.

At that point do we throw up our hands in despair and state that it simply isn't possible? Certainly not! We expand our thought process, put all options on the table, and forge ahead because we must not let technology, particularly of the Oracle variety, defeat us!

Since I was at the point where there was no way to have OpenBD or Railo deal with what the Oracle storedproc was returning (and believe me I'm happy to be proven wrong here), I decided the most expedient route would be to write the database access piece in Java and have CFML call that Java object. Once I'm in Java I can work with the Oracle and JDBC datatypes directly and have more control over what I will return to the CFML engine once I get the REF CURSOR back from Oracle.

The question then becomes if it's better to convert the REF CURSOR into a native CFML query object on the Java side, or to return something like a standard Java ResultSet to CFML and use the ResultSet directly. There are numerous other options as well, particularly once we're dealing with this in Java and have the freedom to do pretty much whatever we want.

I won't go into the gory details, but in one of my early attempts I ran into a problem where I couldn't access the ResultSet because by the time the Oracle delegate class was delegated to the Tomcat delegate class, I had to hack around a bit and get the underlying delegate on the Oracle side just to display the data. Neat problem to solve, but not exactly the most straight-forward solution.

I then decided to try creating a query object native to the CFML engine from the Java ResultSet. This worked well, and is certainly a valid approach, particularly if you know you're going to deploy to a specific CFML engine. If you want to build something that will work on any CFML engine things get more complicated because your Java methods have to return a specific datatype, so you can't have a single method in your Java class that will return the various underlying CFML query object types to the respective engines.

But as a wise man once said, all problems in computer science can be solved by another level of indirection (except, of course, the problem of too many levels of indirection). So I created the Java class to talk to Oracle and return the REF CURSOR, and then wrapped that puppy in a CFC that has conditional logic to return a CFML query object native to the engine on which it's running (i.e. either OpenBD or Railo). Works great!

The only outstanding issue at this point that makes me a hair uncomfortable is that in order for all this to work, I can't explicitly close my callable statements, ResultSets, and datasource connections on the Java side. If I close those items in Java and then return the ResultSet to the CFML engine, then I can't actually iterate over the ResultSet since it's closed.

I could of course close the ResultSet from CFML (I wouldn't have access to the callable statement or the database connection directly), but I'm not nearly as concerned about the ResultSet as I am about the database connection. At least I'm using the Tomcat datasource connection pooling so that should mitigate this concern a bit. In theory Tomcat and the JVM will handle closing this stuff when it's no longer in use, but I need to do a bit of load testing to be sure it's going to work well under load.

There's also some attributes in the Tomcat <Resource> XML definition that you can leverage to alleviate potential problems. Specificaly the maxActive, maxIdle, removeAbandoned, and removeAbandonedTimeout attributes can help keep things cleaned up since I'm in situation where I can't clean things up manually.

As far as the database connection pooling goes, thus far we've seen problems on Windows with Tomcat 6.0.26. It works for a while but then starts throwing database connection pooling errors and all the connections time out. Drop this all on Red Hat Enterprise Linux with Tomcat 7, and things are rock solid. Go figure.

I'm happy to share the Java and CFC code if anyone's interested, and I'm really curious to hear how others have solved this issue because I'm still not convinced what I came up with is the best solution, even though it's certainly working well. So far, anyway (knock on wood).

Apache Error “Document Root Doesn’t Exist” on Red Hat Enterprise Linux When the Document Root DOES Exist

I upgraded one of our Red Hat Enterprise Linux VMs to Tomcat 7.0.4 tonight, did a bit of Apache reconfiguration, and when I restarted Apache I got a "document root doesn't exist" error even though in fact the document root does exist. (Trust me Apache, it's there.)

I double-checked the owners and permissions of all the directories in question and everything was identical to how things are set on another RHEL VM in this cluster on which I haven't upgraded Tomcat and done the reconfiguration yet. I was at a bit of a loss, so I googled around a bit and the prevailing sentiment seemed to be this was related to either A) a config file copied from a Windows box and the line breaks were throwing things off (wasn't applicable in my case), or B) the fact that SELinux was enabled.

If you've been around the Red Hat flavors of Linux long enough you'll remember that SELinux used to be absolutely horrible. For years the very first thing you had to do on Red Hat and Fedora to get anything working at all was turn SELinux off, and for a long time I believe it was even Red Hat's recommendation under their breath to just shut it off. It's gotten better over the years, and honestly stays out of the way to the point where I'd almost forgotten about it.

Since I didn't have anything else to try, however, I went into /etc/sysconfig/selinux and changed SELINUX=enforcing to SELINUX=disabled, restarted the server, and voila the complaining from Apache went away.

What I still don't get is A) why this isn't occurring on my other RHEL box with the same setup, and B) why it just started happening now. The only potential weirdness here is that my document root is a symlink, but again, it's been that way since I setup up these boxes originally and it hasn't been an issue.

So if you run into this same problem the fix (at least until I have more information about why it's happening) is to disable SELinux, but if anyone has more ideas about why this might be happening I'd love to hear them.

Introduction to Tomcat 7 #s2gx

Mark Thomas, SpringSource

• Tomcat 7 Supports …
  • Servlet 3.0
  • JSP 2.2
  • EL 2.2
  • Java 1.6
• New major release of Tomcat every time the spec has a major change
• Servlet 3.0
  • asynchronous processing
  • pluggability
  • annotations
  • session management
  • miscellaneous
• Asynchronous processing
  • request processing is synchronous, but the response processing can now be asynchronous
  • outline
    • start asynch processing
    • request/response passed to a new thread
    • container thread returns to the pool
    • new thread does its work
  • allows container threads to be used more efficiently
    • when waiting for external resources
    • when rationing to a resource
    • or any other time when the container thread would be blocking
  • allows separation of request and response
    • chat applications
    • stock tickers
  • all filters, servlets, and valves in the processing chain must support asynchronous processing
  • not as asynchronous as COMET
• pluggability
  • purpose was to improve developer productivity–worry less about application configuration
  • annotations
  • web fragments
  • static resources in JARs
  • programmatic configuration options
  • pros
    • development can be faster
    • apps can be more modular
  • cons
    • fault diagnostics are significantly hampered
    • might end up enabling things you don't want or need
  • overall, I don't recommend using it for production
  • instead:
    • get tomcat to generate the equivalent web.xml
    • use the equivalent web.xml instead
  • can be frustrating to figure out what's going on when the application is doing things that aren't in web.xml
  • JARs can contain their own web.xml
    • /META-INF/web-fragment.xml
    • essentially same content as web.xml
    • rules for determining fragment ordering
  • allows JARs to be self-contained
  • JARs can also contain static resources
    • always used, cannot be excluded by fragment ordering
    • non-deterministic if there are duplicate reosurces in multiple JARs
• annotations
  • servlets, filters, listeners
    • can be placed on any class
    • tomcat has to scan every class on application start
  • JARs scanned if included in fragment ordering
    • can exclude JARs from the scanning process; controlled in catalina.properties
  • security, file upload
    • placed on servlets
    • processed when class is loaded
  • file upload has almost–but not quite–the same API as Commons File Upload
    • don't have to ship commons file upload with your apps anymore
  • with annotations the configuration can become a lot more opaque
  • can turn all of this off in your main web.xml–turn off metadata complete
    • this is all or nothing–can't pick and choose what bits you want on or off
• programmatic configuration
  • allows a subset of things you can do in we.xml
    • add servlets, filters, and listeners
    • change session tracking
    • configure session cookies
    • configure security
    • set initialization parameters
  • allows greater control / optional configuration
  • some environment-specific settings
  • can make troubleshooting difficult–no xml to refer to in order to see what's going on
  • main advantage is doing things like if/thens in your configuration which you can't do in web.xml
• servlet 3.0 – session tracking
  • adds tracking via ssl session id
    • must be used on its own
  • allows selecting of supported tracking methods
    • url, cookie, ssl
  • url based tracking is viewed as a security risk
    • can't turn this off in servlet 2.2, but can turn it off in servlet 3.0
    • another release of tomcat 6 will likely allow this to be turned off
  • session id is cryptographically secure — can't be spoofed
• servlet 3.0 – session cookies
  • can control default parameters for session cookies
    • name – may be overridden by tomcat
    • domain – may be overridden by tomcat
    • path – may be overridden by tomcat
    • maxage
    • comment
    • secure – may be overridden by tomcat
    • httponly – may be overridden by tomcat
• servlet 3.0 – misc
  • httpOnly
    • not in any of the specs
    • however, widely supported
    • prevents scripts accessing the cookie content
    • provide a degree of xss protection
  • programmatic Login
    • useful when creating a new user account
    • can log the user in without redirecting them to the login page
    • allows the application to trigger a login
• jsp 2.2
  • propery group changes
  • can specify default content type in jsp-config
  • can specify the buffer size for a page
  • new feature – error-on-undeclared-namespace
    • e.g. if you have a typo when using a tag library it fails silently
    • with error-on-undeclared-namespace turned on, error is thrown at compile time
  • jsp:attribute adds support for the omit attribute
• ESL 2.2
  • now possible to invoke methods on a bean
  • correctly identifying the intended method is tricky
  • likely to be some differences between containers–spec if unclear on behavior
  • tomcat tries to do what the java compiler does
• other tomcat 7 changes: management
  • add the ability to fix the remote jmx ports
    • previously jmx picked a port at random
  • single line log formatter
  • manager app can distinguish between primary, backup, and proxy sessions (for clusters)
  • aligned mbeans with reality (GSoC 2010)
  • general improvements to JMX support
    • can now have a server.xml with just a <Server …/> element and create a fully working Tomcat instance (Hosts, Contexts, etc. all via JMX)
      • can't save this config out but that's being worked on
• performance
  • unlikely to see a big change
  • can limit the number of JSPs loaded at any one time
    • useful for development
  • not many areas where tomcat needs a big performance boost
• security
  • generic CSRF protection
    • if you go to a site with malicious code, might trigger your browser to make a call to the tomcat manager to deploy an app that gives access to your machine
    • now the manager looks for a token that was passed from the previous response to the manager app and if the token doesn't exist, the request will fail
  • separate roles for manager and host manager apps
  • session fixation protection
    • changes session ID on authentication
  • enable the LockOutRealm by default (e.g. lock out user for 10 minutes after 5 failed login attempts)
  • enable an access log by default
  • added ability to disable exec command for SSI
• code cleanup
  • use of generics throughout
  • removed deprecated and unused code
  • reduced duplication, particularly in the connectors
  • better definition of the lifecycle interface
  • added checkstyle to the build process
  • if you've written your own custom tomcat components, you might need to change them for tomcat 7
• extensibility
  • added hooks for rfc66 – used by virgo
  • refectored to simplify geronimo integration
  • significantly simpler embedding
• stability
  • builds on tomcat 6
  • tomcat 6 is already very stable
  • significant reductions in the open bug count
    • 6 open bugs without patches when i wrote this slide
    • for tomcat 5.5.x, 6.0.x, and 7.0.x combined
  • added unit tests
    • CI using BIO, NIO, and APR/native on every commit
  • memory leak detection and prevention
    • back-ported to tomcat 6
• flexibility
  • copying of /META-INF/context.xml is now configurable — can control whether or not the expansion/copying of this file happens
  • alias support for contexts
    • map external content into a web application
    • keeps tomcat from deleting things in a symlink when the app is undeployed
  • shutdown address is now configurable
    • deliberately limited to localhost by default
  • tomcat equivalent of some httpd modules
    • mod_expires
    • mod_remoteIP
• tomcat 7 status
  • passes servlet 3.0 TCK with every combination of connectors
  • passes jsp 2.2 TCK
  • passes EL 2.2 TCK
  • all with the security manager enabled
  • note that just because it passes the TCK doesn't necessarily mean it's fully compliant
  • 7.0.4 just released today
• when will tomcat 7 be stable?
  • when three +1 votes come from committers
  • in practice the committers each have their own criteria
  • i'm looking for 2-3 releases with …
    • no major code changes that might cause regressions
    • tcks all pass (already have this)
    • no major bugs reported
    • good levels of adoption (already have this)
• tomcat 7 plans
  • one release every month
    • bug 49884 put a spanner in the works
  • stable by the end of the year?
  • keep on top of the open bugs
  • work on bringing the open enhancement requests down
  • if all goes well, 7.0.6 will be the stable release
  • jsr 196 implementation?
    • authentication SPI for containers
    • geronimo has most (all?) of this already
  • windows authentication
    • looking unlikely — too much baggage
      • needs some native libraries for it to work well
    • waffle project already does this
  • simpler jndi configuration for shared resources
    • no more <ResourceLink … />
  • more jmx improvements
  • further improvements to memory leak protection
  • continue migration from valves to filters
  • java ee 6 web profile
    • no interest so far from user community
    • had more questions from journalists than users
    • no plans at present
    • adds a lot of baggage that isn't that useful
    • if you want a web profile implementation, there's geronimo
• useful resources
  • http://tomcat.apache.org
  • http://blogs.apache.org/tomcat
  • mailing lists
    • users — very busy
    • dev
    • announce
  • http://tomcat.markmail.org
  • http://svn.apache.org/repos/asf/tomcat/trunk/webapps/docs/changelog.xml
  • http://ci.apache.org/projects/tomcat/tomcat7/docs
  • http://s.apache.org/tomcat-7-open-issues
• new feature — rolling update/side-by-side deployment
  • can deploy a new version while the app is running and when a user's session expires, they hit the new version of the app
  • came out of a tc server requirement but made more sense to implement it in Tomcat
  • springsource providing patch to ASF and will be part of a future tomcat release
  • deploy a new WAR with the same name as an existing app, but add ##N at the end of the war file name where N is the version (e.g. myapp##1.war will be a new version of myapp.war)
    • context path is retained, meaning context path is the same for both versions of the app
  • feature that will be added is when no more sessions are active on the old version it will be automatically undeployed

Clustering and Load Balancing With tc Server and ERS httpd #s2gx

Mark Thomas – SpringSource

• Tomcat committer
• tc Server developer
• responsible for keeping tc Server and Tomcat in sync
• memory leak detection in tomcat manager app
• recent logging improvements
• simplifying jmx access
• all of the above started in tc Server, but have been contributed back and implemented these features in tomcat
• don't want to get into having a significant fork of tomcat

Typical Architectures

• load balancer (round robin) -> httpd (sticky sessions) -> tc Server (clustered)
• don't go anywhere near tc Server clustering unless you absolutely have to–adds complexity and overhead
• only thing tc Server clustering gives you is the ability for users not to lose sessions if an instance of tomcat goes down
• ask yourself how big of a deal it is if your users lose their sessions when an outage occurs–if it's a big deal then you may need clustering

Starting Point

• ubuntu 8.04.4 64-bit VM
• vmware tools installed
• 64-bit sun jdk 1.6.0_21
• will be installing tc Server, Hyperic, etc. on this clean image

tc Server Installation

• don't run tc Server as root
• create a tcserver user
• owns the tc Server files
• runs the tc Server processes
• install to /usr/local/tcserver

Instance Naming and Port Numbering

• think about this in advance–may wind up with 100s of instances
• tc01, tc02, etc. as the instance name, then follow this for ports
• example scheme for ports
• 1NN80 – http
• 1NN43 – https
• 1NN09 – ajp
• 1NN05 – shutdown (if used)
• 1NN69 – jmx
• server and jvmRoute naming–consider linking server name to IP address, e.g. srvXXX-tcYY where XXX is the end of the IP address, YY is the tomcat instance number
• 1NN20 – cluster communication

DEMO: Installing tc Server

• tc Server version names are e.g. apache-tomcat-6.0.29.A.RELEASE where the first part is the version of Tomcat, the "A" means it's the first release of tc Server based on that tomcat release
• if shutdown port is disabled, doing a kill -15 does a graceful shutdown. kill -9 works too and tomcat won't care, though your application might, so only do -9 if you have to
• created two instances of tc Server using the tc Server create instance script
• tc Server comes with templates for startup scripts–copy these over to /etc/init.d and edit as needed
• paramterize cluster addresses and ports in a catalina properties file
• can use ${…} notation in server.xml to hit the properties in catalina.properties

Creating a Cluster

• switching to static node membership
• cumbersome for large clusters
• remove the <Membership …/> element
• need to add a bunch of config stuff after the <Interceptor …/> elements
• easier to use dynamic node discovery
• backup strategies — tomcat gives you DeltaManager and BackupManager
• delta manager is simplest–replicates every session to every node in the cluster
• if your sessions use a lot of memory, delta manager doesn't give you much scalability
• if your limitation is CPU, delta manager gives you some scalability
• amount of network traffic on delta manager increases with the square of the number of nodes–not terribly scalable
• backup manager
• replicates session data to one other node in the cluster
• send options: synchronous vs. asynchronous
• in synchronous, writes session changes to other nodes, waits for acknowledgement, and then sends response to the user. can mean a lag for the user.
• asynchronous — changes to sessions are put on a queue and the user gets the response immediately. means there's a chance that the cluster will be in an inconsistent state. use of sticky sessions means the consistency of the cluster doesn't really matter.
• because java thread running isn't deterministic, in asynchronous mode the session updates may not be processed in the same order in which they were placed on the queue, so if your application depends on these being processed in the same order this is a risk
• no need for the WAR farm deployer — hyperic does this better
• WAR farm deployer has been removed from tc Server
• backup manager DOES know where the primary and backup nodes ARE for every session
• i.e. it doesn't actually store all the sessions from all nodes, but it knows where to get the session it lost
• backup manager scales much better than delta manager in both memory and network traffic
• network traffic scales linearly with number of nodes
• for availability on a small cluster, use the delta manager
• if you're worried about scalability, go with the backup manager

Hyperic HQ Installation

• create an hqs user
• hqs user owns the hyperic hq agent files
• the agent itself runs as the tcserver user
• os security considerations
• agent doesn't need root privileges to access OS mechanics, start/stop processes, etc.
• tc Server needs to be able to read WAR files uploaded via the agent
• don't want tc Server runtime running as root
• hyperic security considerations
• don't want agent connecting as hqadmin super user
• create a dedicated agent user
• requires create, modify, and delete privileges for platform and platform services only

ERS httpd

• ERS = Enterprise Ready Server
• SpringSource's distribution of Apache httpd
• install ERS as root
• httpd processes run as nobody:nobody so this is fine
• remove the test instance
• create a new instance
• module configuration
• enable mod_proxy_balancer
• enable mod_proxy_ajp
• mod_proxy_ajp isn't quite as stable vs. mod_jk and mod_proxy_http
• mentioned something about mod_http now having remote IP addresses available–need to ask about this
• configure balancer in ers


<Proxy balancer://tc>
  BalancerMember http://ip.address.here:port route=tc01-uniqueID
  BalancerMember http://ip.address.here:port route=tc02-uniqueID
</Proxy>

ProxyPass /cluster-test balancer://tc/cluster-test stickysession=JSESSIONID:jessionid
ProxyPassReverse /cluster-test balancer://tc/cluster-test


Debugging Clusters

• need something in your apps that tells you which cluster node you're on
• also need something to spit out the session ID so you can test that the sticky sessions are working
• if your context path differs from your host name in tc Server, this may cause your cookies not to work since the hosts are different
• can use cookiepath in proxypassreverse directive
• easier: just have your context path match your host name
• anything you want replicated in sessions has to be serializable
• if your application can't support having everything in the session be serializable, terracotta will support non-serializable data in session replication

Gaining Visibility Into Enterprise Spring Applications with tc Server Spring Edition #s2gx

Gaining Visibility Into Enterprise Spring Applications with tc Server Spring Edition
Steve Mayzak – SpringSource

• tc Server — enterprise version of Tomcat developed by SpringSource
• Built on Tomcat
• Survey in 2008: 68% of companies surveyed using Tomcat; most popular lightweight container

tc Server editions

• developer edition — can get it when you download STS
• standard edition — application provisioning, server administration, advanced diagnostics
• spring edition — spring + tomcat stack, spring application visibility, spring performance management

tc Server: Key Highlights

• Developer efficiency
• operational control
• deployment flexibility
  

Spring Insight

• Spring Insight — knows about Spring and Grails applications, so can provide specific information about your apps as they're running
• When you deploy a WAR to tc Server, Spring Insight gets involved and get get specific information from your apps
• Spring Insight is currently intended for development use only
• DEMO: huge amount of details come out of Spring Insight, from the HTTP request details down to the JDBC statements that were run and timings on the various actions during the request
• can use Insight with other frameworks, etc. as well by adding annotations to your code

Operational Control

• performance & sla management of spring apps
• application provisioning and server administration
• rich alert definition, workflows, and control actions
• group availability & event dashboards
• secure unidirectional agent communications
• tc Server is a combination of Hyperic and Tomcat
• monitoring is done via valves in Tomcat–isn't a fork or modified version of Tomcat
• Hyperic monitors web servers, app servers, databases, caching, messaging, directories, virtualization, etc.
• Hyperic is also a management tool–admin, provisioning, groups, metrics, alerts, events, access control, upgrades, etc.
• hyperic is jmx based, runs as an agent on each server

Enterprise Capabilities in tc Server

• Run multiple instances per install — creates tc Server install updates
• tc Server instances can point to a central set of binaries so upgrades are simpler
• advanced scalability options
• non-blocking (NIO) conectors
• high-concurrency connection pool
• JDBC Pool — open source (ASF), blog posts on tomcatexpert.com
• Can create tc Server templates and create multiple instances from a base template
• Advanced diagnostics–detects deadlocks and slow-running requests

Hyperic

• monitoring and deployment
• can see what apps are running on tc Server, number of sessions on the apps, up/down times, etc.
• can deploy war files and change context path as you deploy
• can schedule deployments and have the server auto-restart
• can deploy from a remote machine (e.g. build server)
• can remotely stop/start/restart instances
• can set up scheduled restarts–e.g. make configuration changes during the day, have server auto-restart after hours
• can schedule recurring restarts (e.g. restart daily at 1 am)
• hyperic consumes jmx metrics and can show them in context, meaning it shows application metrics in the context of the overall server health (cpu, memory, etc.)
• with the metrics coming in you can look at the specifics of your application and tune according to what your specific application is doing
• has everything from cpu, memory, disk I/O metrics, to request metrics, down to spring-specific metrics
• can add JMX instrumentation to your own classes without having to write your own MBeans–just annotate with @ManagedResource, @ManagedMetric, @ManagedOperation, @ManagedAttribute
• this lets you get down to questions like "how many bookings per second can I handle in my travel booking application?" which lets you plan infrastructure and scalability in a very granular way
• hyperic does metric baselining for your specific app so you'll get alerts based on the baseline metrics for your application
• can enable/disable metrics and alerts across multiple servers from a single interface
• can see metrics globally (across a cluster) or individually on each server

Deployment Flexibility

• Lean server (10MB) for virtual environments
• template-drive server instance creation
• integrated experience with vmware environments
• open, secure API for all operations
• server-specific settings like ports, etc. have been taken out and put into a catalina properties file, so server.xml can be applicable to all servers
• streamlines process of spinning up new server instances
• shared binaries for upgrades
• multiple server versions can be installed per machine
• complete flexibility for various "sizes" of VMs

—
Matthew Woodward
matt@mattwoodward.com
http://blog.mattwoodward.com
identi.ca / Twitter: @mpwoodward

Please do not send me proprietary file formats such as Word, PowerPoint, etc. as attachments.
http://www.gnu.org/philosophy/no-word-attachments.html

Accessing a Network Drive from Apache and Tomcat on Windows Server

A few quick tips if you find yourself having to access a network drive from Apache and Tomcat on a Windows Server. This is all pretty much old hat but since I still get questions about this fairly regularly and was just setting up some things on some servers this weekend, I figured I’d write this up.

In my situation we’re running an application on three VMs behind a load balancer, and users can upload files from the application. Since I didn’t want to set up a process behind the scenes that copied uploaded files between all three servers (though it would have given me a chance to take another run at using Unison), we have an uploads directory living on a SAN. This way no matter which VM the user is on, the uploads all go to and are served from a single file system.

On Windows Server, by default services run under the Local System Account, which doesn’t have access to network resources. So if you install Apache and Tomcat as services and expect to be able to point them to a UNC path for some of your content, that won’t work out of the box. You need to be running Apache and Tomcat under an account that has network access. In most environments in which I’ve worked this type of account is typically called a “service account,” because you’ll end up getting a domain account just like your typical Active Directory user account, but of course it won’t be associated with a human being.

Once you have that account in place, you go into your services panel, right click on the service, click on “Properties,” and then click the “Log On” tab. You’ll see by default the Local System Account radio button will be checked. Click the radio button next to “This Account,” enter your service account information, click “OK,” and then restart the service. At this point your service will be running under the service account and will have access to network resources. Note that you’ll have to do this for each service that needs access to the network drive, which in my case meant doing this for both Apache and Tomcat.

That takes care of the web server and things at the Tomcat level in terms of basic access, but you’ll likely be configuring an alias of some sort to point to the network drive. In my case I wanted /uploads to point to \serversharenameuploads, which meant setting up an alias in Apache, a Context in Tomcat, and a mapping in OpenBD. This is where a lot of people get confused, so I’ll go through each of these settings one by one.

The necessity for a web server alias is probably pretty obvious. If you’re serving an image directly from your web server, e.g. http://server/uploads/image.gif, if /uploads doesn’t exist under your virtual host’s docroot, then Apache will throw a 404.

Allowing Apache to access the network drive involves (depending on how you have Apache configured) using a Directory directive to give Apache permission to access the directory, and then an Alias directive so Apache knows where to go when someone requests something under /uploads. So the following goes in your virtual host configuration file:


<Directory "\serversharenameuploads">
  Order allow,deny
  Allow from all
</Directory>

Alias /uploads "\serversharenameuploads"


You may have other stuff in your Directory directive as well but that’s the basics of what will allow Apache to see /uploads as the appropriate location on your SAN.

The next layer down will be your CFML engine. Remember that if in your CFML code you want to read or write files to /uploads, even though Apache knows what that is now, your CFML engine will not. I’m emphasizing that point because it’s such a common source of confusion for people. If things are happening in your CFML code, it won’t be interacting with Apache at all, so it won’t know about the Alias you set up in Apache. Simple enough to solve with a mapping; just go into your CFML engine’s admin console and create a mapping that points /uploads to \serversharenameuploads and that handles things at the CFML level.

Lastly comes Tomcat. Depending on how you’re serving files, you may be proxying from Apache to Tomcat, so if Tomcat needs to know where /uploads lives, since it’s not in the webapp’s base directory Tomcat will throw a 404 unless you tell it where /uploads is located.

Tomcat doesn’t have Aliases in the same way Apache does, but what you can do in Tomcat is configure multiple Contexts under a single host. So in Tomcat’s server.xml (or in a separate host config file if you prefer), you simply add a Context that points /uploads to the network location:


<Host name="myapp">
  <Context path="" docBase="C:/location/of/my/app" />
  <Context path="/uploads" docBase="\serversharenameuploads" />
</Host>


So now you have things set up in such a way that Apache, your CFML engine, and Tomcat all know where /uploads really lives.

Another point of confusion for people on Windows is the concept of “mapped drive” letters. A lot of people think that if you map \serversharename to a drive letter of let’s say D:, than in your code you can then access the uploads directory we’ve been using as our example via D:uploads.

The simplest way to explain why this doesn’t work is to point out that mapped drive letters are associated with a Windows user account. They don’t exist at the operating system level. While you may remote into the server using your credentials, map to a network location, and assign that to drive letter D:, another user logging in won’t see that mapping, and services running on the server under various user accounts definitely won’t know anything about mapped drives.

This is why in all the examples above you see the full UNC path to the network resource being used. You have to use the UNC path in order to get this all to work correctly because that’s the only way services running under a service account will be able to address the network resource.

Hope this helps eliminate some of the persistent confusion I see around this issue.

Quick Apache/Tomcat Proxying Tip

I’m setting up some new servers with some semi-complex URL rewriting and proxying going on of both the HTTP and AJP varieties. For this particular application the home page is actually a completely separate application that’s running as its own webapp on Tomcat, and everything else runs from within the webapp itself. (I’ll spare you the details; just assume I’m not insane and this was the best way to handle this situation.)

So for all this to work properly I have the following rewrite rule:

RewriteRule ^/$ http://otherapp/ [NC,P]
ProxyPassReverse / http://otherapp/


Then below that I have a bunch of AJP rewrites for the webapp itself. What was throwing me fits was that everything was working great except for this one rewrite rule, which would attempt to load indefinitely until I killed the request.

I stared at my virtual host config file for quite a while and finally paused on this line:

ProxyPreserveHost On


Given that the ^/$ rewrite rule is proxying out to a different host, it seems kind of obvious now that of course that would confuse Apache. Removing that line and adding some aliases for the other host’s CSS and images directory did the trick.

Maybe no one else will be in this exact situation, but if you’re ever faced with a blank white screen while waiting forever for an HTTP proxy to go through, this tip might come in handy.

Tomcat 7 Released

Apache Tomcat 7.0, the latest version of the popular open source Java Web server, is available Tuesday from the Apache Software Foundation.

The upgrade constitutes the first major release of the project since 2006. To ease Web framework integration, version 7.0 implements the Java Servlet 3.0, JavaServer Pages 2.2 and Expression Language 2.2 specifications.

via networkworld.com

Big changes are the support of Servlet 3.0, JSP 2.2, and EL 2.2, but from what I’ve read there are lots of nice fixes and performance improvements as well. I’m about to set up two new servers so this is great timing!

Full details at https://blogs.apache.org/tomcat/entry/apache_tomcat_7_released

A New Apache Tomcat Community: TomcatExpert.com | Javalobby

A new site launches today aiming to be the definitive resource for developers who want to run Apache Tomcat in large scale production environments. Sponsored by SpringSource, tomcatexpert.com will provide a central point for the Tomcat community.

via java.dzone.com

Just launched today but given that it’s being sponsored by SpringSource, I’m sure this will evolve into a fantastic resource very quickly.