I don't use Eclipse, but there's not all that much special here with Windchill.
The standard "classpath" for a JSP is dictated by the J2EE JSP and servlet specifications and boils down to <webAppDocumentBase>/WEB-INF/classes and <webAppDocumentBase>/WEB-INF/lib/*[.jar]. For taglibs, etc, there is also special handling based on what is found in WEB-INF/web.xml. Various file references are relative to the web application's document base.
In the case of Windchill, <webAppDocumentBase> is WT_HOME/codebase. This itself is at the front of the classloader lookup, rather than <webAppDocumentBase>/WEB-INF/classes. Also in 10.0 and higher, the classpath is augmented with WT_HOME/lib/*[.jar] -- and whatever else is added to the Windchill server classpath. Apart from those classpath oddities, however, Windchill just follows normal JSP/servlet development.