I recently had trouble with a web application deployed on Tomcat that leaked its ClassLoader every time it was redeployed resulting in OutOfMemoryErrors after a few redeployments. This is quite nasty if you plan to do continuous deployment and don’t want to restart the servlet container with each deployment.
Recent versions of Tomcat include some code that makes you aware of problems when you undeploy the application:
SEVERE: The web application [] registered the JDBC driver [oracle.jdbc.OracleDriver] but failed to unregister it when the web application was stopped. To prevent a memory leak, the JDBC Driver has been forcibly unregistered. SEVERE: The web application [] appears to have started a thread named [Thread-14] but has failed to stop it. This is very likely to create a memory leak. SEVERE: The web application [] appears to have started a thread named [Thread-15] but has failed to stop it. This is very likely to create a memory leak. SEVERE: The web application [] appears to have started a thread named [Thread-16] but has failed to stop it. This is very likely to create a memory leak. SEVERE: The web application [] appears to have started a thread named [Thread-17] but has failed to stop it. This is very likely to create a memory leak. SEVERE: The web application [] appears to have started a thread named [Thread-18] but has failed to stop it. This is very likely to create a memory leak.
As you can see Tomcat managed to unregister the JDBC driver that the application had failed to unregister but could do nothing regarding the threads that had been started but not stopped.
I ran the application with YourKit attached to check that the WebappClassLoader had actually leaked and to see what those threads were that prevented it from being garbage collected. The “Paths from GC Roots” view in YourKit is well suited for this:
As you can see there are four threads from ONS that prevent the ClassLoader from being garbage collected: oracle.ons.SenderThreads and oracle.ons.ReceiverThreads.
I wrote a small ServletContextListener that shuts down ONS to get rid of them. After that I noticed that Oracle registered a OracleDiagnosabilityMBean that I had to unregister. Finally I made sure the JDBC drivers that the application had registered were properly unregistered from DriverManager.
With those changes in place the application undeployed well and was fully garbage collected.
Here is the code:
import oracle.ons.ONS;import oracle.ons.SenderThread;import org.slf4j.Logger;import org.slf4j.LoggerFactory;import org.springframework.util.ReflectionUtils;
import javax.management.MBeanServer;import javax.management.ObjectName;import javax.servlet.ServletContextEvent;import javax.servlet.ServletContextListener;import java.lang.management.ManagementFactory;import java.lang.reflect.Field;import java.lang.reflect.Method;import java.sql.Driver;import java.sql.DriverManager;import java.util.ArrayList;import java.util.Enumeration;import java.util.Hashtable;import java.util.List;
public class CleanUpListener implements ServletContextListener { private Logger logger = LoggerFactory.getLogger(getClass());
@Override public void contextInitialized(ServletContextEvent sce) { // do nothing }
@Override public void contextDestroyed(ServletContextEvent sce) { shutdownOns(); deregisterJdbcDrivers(); }
@SuppressWarnings("unchecked") private void shutdownOns() { logger.info("Shutting down ONS"); final Method getRunningONS = ReflectionUtils.findMethod(ONS.class, "getRunningONS"); final Method shutdown = ReflectionUtils.findMethod(ONS.class, "shutdown"); ReflectionUtils.makeAccessible(getRunningONS); ReflectionUtils.makeAccessible(shutdown); final ONS ons = (ONS) ReflectionUtils.invokeMethod(getRunningONS, null); if (ons == null) { return; } ReflectionUtils.invokeMethod(shutdown, ons);
final Field senders = ReflectionUtils.findField(ONS.class, "senders"); ReflectionUtils.makeAccessible(senders); final List<SenderThread> senderThreads = (List<SenderThread>) ReflectionUtils.getField(senders, ons); if (senderThreads == null) { return; } final Method stopThread = ReflectionUtils.findMethod(SenderThread.class, "stopThread"); ReflectionUtils.makeAccessible(stopThread); for (SenderThread senderThread : senderThreads) { ReflectionUtils.invokeMethod(stopThread, senderThread); } } private void deregisterJdbcDrivers() { logger.info("Deregistering JDBC Drivers"); final Enumeration<Driver> driverEnumeration = DriverManager.getDrivers(); final List<Driver> drivers = new ArrayList<Driver>(); while (driverEnumeration.hasMoreElements()) { drivers.add(driverEnumeration.nextElement()); }
for (Driver driver : drivers) { if (driver.getClass().getClassLoader() != getClass().getClassLoader()) { logger.debug("Not deregistering {} as it does not originate from this webapp", driver.getClass().getName()); continue; } try { DriverManager.deregisterDriver(driver); logger.debug("Deregistered JDBC driver '{}'", driver.getClass().getName()); if ("oracle.jdbc.OracleDriver".equals(driver.getClass().getName())) { deregisterOracleDiagnosabilityMBean(); } } catch (Throwable e) { logger.error("Deregistration error", e); } } }
private void deregisterOracleDiagnosabilityMBean() { final ClassLoader cl = Thread.currentThread().getContextClassLoader(); try { final MBeanServer mbs = ManagementFactory.getPlatformMBeanServer(); final Hashtable<String, String> keys = new Hashtable<String, String>(); keys.put("type", "diagnosability"); keys.put("name", cl.getClass().getName() + "@" + Integer.toHexString(cl.hashCode()).toLowerCase()); mbs.unregisterMBean(new ObjectName("com.oracle.jdbc", keys)); logger.info("Deregistered OracleDiagnosabilityMBean"); } catch (javax.management.InstanceNotFoundException e) { logger.debug("Oracle OracleDiagnosabilityMBean not found", e); } catch (Throwable e) { logger.error("Oracle JMX unregistration error", e); } }}
