HumaniNet began working with Plone in early 2005 to accomplish three primary goals:

1. Streamline our editorial process for information we share publicly and with field partners (Publishing)
2. Share research notes, links, meeting minutes and other internal information (Intranet/file-sharing)
3. Discuss and collaborate on projects - both internal and joint efforts with outside organizations
   

If you are interested in the first phase of the project, during which we prototyped the site with Plone 2.0.5, please review part one of our lessons learned.

By the summer of 2005, we had reached a plateau with Plone 2.0.5.  One option was to plow ahead towards our goals, which would have required us to spend significant amounts of the remaining project funding building around some of the problems we had with that version. An alternative was to wait to see what was included in the final release of Plone 2.1, which was due to be released in August 2006, and build our site on the new version.  We waited, and that turned out to be the right decision. 

Plone 2.1 had significant improvements, such as live search, better editing, related items linking, smart folders, more flexible display options, and better performance.  Making our customizations on top of this version will significantly extend the useful life of our present site.

By this time, we had also settled on SalesForce.com as our primary tool for managing information about people: field partners, volunteers, donors, contacts, and mailing lists.  Trying to link this data directly into Plone, for authentication and content segmentation, become a fourth goal of this phase of the project.

Before getting into the specifics, I want to reiterate that any criticisms or frustrations expressed below should be read within a context of deep appreciation for all the hard work of Plone developers around the world. My goal for this document is to explain our experience to  other non-profits, in hopes that it will save them time and effort in evaluating content management systems and using Plone.

I should also clarify that this is a summary of our experiences over the last few months, and is not intended as a comprehensive evaluation of Plone as a platform. There are solutions, I am sure, for many of the technical problems described below. We have a very long list of unresolved issues, and this document describes where we were at the end of the project in April.

Adding/Editing Content

• The default WYSIWYG editor in Plone 2.1 is now Kupu, which is an improvement over the old default, epoz.  Table editing and formatting are much better, the external linking window has a preview area, and the internal linking window displays a navigable view of all the content items on the site. Unfortunately, it doesn't seem possible to link to folders directly, only pages and news items, etc.
  
• At times, some browsers display the the scroll bar of the edit area on top of the link preview panes.  While this is annoying and looks strange, it does not prevent the creation of links.
• Plone 2.1 asks users if they want to save their work whenever a user clicks cancel or tries to navigate away from an item that has been edited but not saved.  This is a very useful feature.
• Having multiple editors of a single content item continues to be a concern. It is simply too easy for one user to save over another's changes.  There is currently no user-accessible versioning system in Plone, so in this scenario, prior changes are lost. Technically speaking, the 'lost' versions are in the ZODB, but in practical terms, I'm not aware of any way to easily retrieve them.
  
• Because of this, I have recommended that our Plone installation should not be used as a primary source for critical documents or document changes.  Instead, documents should be composed and edited offline or using other tools, and then saved for additional comment or future reference within Plone.
• We tried the Locking Workflow product, but the results were inconsistent across different user accounts, and at some point during our testing, it just seemed to stop working.
• We also recommend that only the original author should edit an item, while others just add comments.  In our installation, this guideline is not strictly enforced.  The workflow could be altered so that only owners and managers have edit permissions on content items, but we have a number of use cases in which we need to allow non-managers other than the original author to edit a document.
• Even if an item has only one editor, it is easy to lose critical information. For example, if you go into the editor, select all body text, hit the delete key, and click save, you have just essentially lost the content, even though a past version is buried inside the ZODB somewhere.
• The behavior of the undo feature is variable, and should not be relied upon to recover critical information.
  

Notification/Attention

• The lack of customizable, user-oriented notification has been one of our biggest frustrations during this project.  We have a number of requirements: that each user can specify if and how often they receive notifications, that the notifications can be filtered to a specific project or topic, and that users can adjust these settings on their own.
• The PloneSubscription product seems to offer some, but not all of these features, as of the last review I made of it, but we simply haven't had time to fully test it.
• Plone has a very robust RSS generation system, but for the most part, our user base is still not "RSS-ready", even if the technology is great. External RSS-to-email services like FeedBlitz and FeedBurner Email may be useful alternatives to direct consumption of syndication feeds.
  
• User-initiated email would be another option.  In Basecamp, for example, the edit screen for messages include checkboxes to send email notifications to everyone else on a project team, or selected members. In Plone, using the email icon displayed on each page is not effective for group notification, because email addresses need to be entered manually. The user would have to know who to send the message to, and repeat the process for each person. Since users would be typing addresses rather than picking from a list, data entry errors are likely, and any bounces go to the site manager's inbox, not the original sender.  This is also not feasible for more than 3 or 4 people, or when all participants aren't known. 
  
• We are using the GrufSpaces (aka GroupSpace) product for our project spaces, and the email tab simply doesn't work because of a permissions problem. Also, the only place to see a list of member is on the Roles tab, but anyone with the access to the Roles tab can add or delete members and permissions for that area.  There is no read-only view, so there is no way for non-leaders to even see who else is in the group unless a manual list of participants is maintained by the project leader.
• As a substitute for these other strategies, I implemented some wiki-esque hacks: Using smart folders, a much improved version of what was known as Topics in Plone 2.0.5, I created two lists: recent changes and recent comments.  These lists are generated in real-time, and are permissions-based - i.e., they only show the items that a particular user has permissions to see.
• In our prototype, we had recent items and recent comments as portlets, which only list the titles and last modified dates of five or so headlines.  Using smart folders accessible via tabs at the top of the site has been much more effective.
  
• The list of recent comments still lacks context in that it shows all comments across the entire site, without any path or location information.  I added location into the smart folder results in Plone 2.1.1, and it was always blank.  This has not be re-tested.
  

User Interface

• Plone 2.1 changes the default view template to display the real name of the author rather than their user name.  This is a great improvement.
• Comments still use the username, e.g. mblair instead of Matt Blair, which can make them more difficult to read.
• To notify users of unresolved problems and the workarounds for them, we created a Known Issues page in the help folder, with a tab that links to it.
• Reordering items in the contents view of a folder still requires repeatedly clicking on the tiny little up and down arrows. One alternative in Plone 2.1 is to specify a page or other content item as the default display for a folder, instead of the standard folder listing.  We have used this to create a 'managed' index that can be more easily reordered and edited, but you lose the automated listing features.
• After upgrading our site to Plone 2.1.2, we encountered a new error that displays a 404/"Page not found" error if a user tries to access a deep link to workflow-protected without first logging in. This has something to do with the way the destination URL is passed along during log in, but we have not been able to determine the exact cause.
• Plone 2.1.2 also has a bug that causes the main body of a page to be completely blank if it doesn't like something about a non-standard portlet. A custom portlet that I created to list comments that were local to the current folder triggered this problem.  Removing the portlet fixed the problem, but we no longer have a list of local comments.
• Plone's favorites are no longer enabled by default, but very few of our users were using them, so we did not re-implement this in the second phase.
• Language support continues to be good, and Plone 2.1 introduced right-to-left support for languages like Arabic.  The core user interface elements of Plone have been translated into over 50 languages, and the language for these elements is chosen automatically based on browser/computer settings.  For example, if you set the default language of your browser to Turkish, and go to a site based on Plone 2.1, all the basic elements of the user interface will automatically appear in Turkish.
• In our testing, parallel translation of content was a little more problematic.  While using an early release of LinguaPlone (we were using 0.8.0) on a test instance, an entire containing folder of content become unreadable.  To be fair to them, it was labeled a beta, and I have heard that recent versions are much more stable, but we have not had a chance to re-evaluate.

Configuration and Setup

• Setting up our instance of Plone was far from point-and-click, as our step-by-step how-to shows.
• In Plone 2.1, there is still no easy way to copy all the configuration details of one Plone instance and apply them to another, or to clone an installation and then fill the new site with different content and users. This is possible with customization policies, which would involve translating all configuration changes into Python scripts. Designing and testing these scripts is beyond the capacity of our organization, and this sort of investment probably only makes sense if the cost can be spread across many Plone sites that are heavily customized in similar ways. This news item indicates that the situation should improve with an XML config export in Plone 2.5, which is due for release in May or June 2006.
  
• Our solution was to put all of our content and activity into one Plone instance, instead of trying to maintain a number of different Plone sites at the same time. Our content is divided into GroupSpaces, which are managed by a custom workflow.  This approach lets us partition access to all content based on group memberships and role assignments at both the site and folder levels.  While managers will see a dozen or more folders in the navigation portlet when they log in, other site users may see only one or two.  
• We have explicitly blocked nested GroupSpaces, to try to avoid unnecessarily complicated acquisition scenarios.
  

Management and Administration

• As mentioned in part one of our lessons learned, many configuration tasks must be performed in the Zope Management Interface (ZMI), and there is no effective audit trail or rollback.  This has not changed significantly with Plone 2.1. A simple change to the position of portlets, for example, can only been done in the ZMI. 
  
• Here's another example: Plone 2.1, in its default configuration, creates a tab across the top for each top-level folder.  In our instance, we have disabled this feature - with dozens of top-level folders available to our core team, every page would scroll horizontally for several screens if we didn't.  We have a few basic tabs set up now, but creating any new tabs requires access to the ZMI.
• Within the ZMI, there is no way to limit access to specific areas.  This makes it extremely difficult to delegate administrative tasks, because letting 'just anyone' into the ZMI is simply too dangerous. A Plone administrator will also need access to the Zope instance via secure FTP and SSH, and possibly even root access on the server, depending on how the server is configured. These are not credentials that should be handed out lightly.
  
• Once we made our site private, only those assigned the Manager role could add users. It would be nice if there was a way to add a user without exposing the entire site's role and group assignment pages.  Intentionally or not, a user could seriously damage the security of a Plone site by altering the assignment of roles to users within the Users and Groups section of the Plone control panel.  The alternative is to make a user with manager permission add every single user manually.
  
• We tested the CMFMember product in this phase, which can put members through an approval workflow, but we found significant problems with it.
  
• The GroupSpaces Roles tab lets us delegate some project membership tasks to project leaders, but a manager still needs to create an account first.
• In phase two of this project, we migrated parts of the site twice, and finally aggregated all our Plone-based content into this existing site, which is based on Plone 2.1.2.  The migrations can be difficult, especially if third-party products are involved.  We did an in-place migration from Plone 2.1.1 to Plone 2.1.2, and none of the content within Groupspaces would display properly until it was re-imported from backups and put through a workflow transition.
• In general, small organizations will need to be selective about which migrations to perform, and should plan to run the whole migration on a test server to identify any major problems in advance.  Sometimes re-indexing the catalog in the ZMI will recover items that seem to be lost. It may be helpful to setup a parallel instance of Zope/Plone rather than migrating in-place on the same Zope instance.
  
• Plone is stores data in the Zope Object Database (aka ZODB), which can be difficult to maintain for those accustomed to relational databases - which is just about everybody that works with databases. There is no backup button in the ZMI. The ZODB can be copied on the file system level, or a script called repozo can be used to backup ZODBs in an internally consistent state. Using repozo to manage backups can be considerably more complicated that performing similar functions with a relational database like PostGreSQL.
  
• Would it be possible to go in and retrieve a single past version of a document? This is a request that is all too familiar to anyone who has ever administered a document repository or file server, but I haven't seen a simple, clearly documented way to do this in the ZODB.
• As part of the re-design of our workflows, we added an archive state for content that is no longer needed on the site.  Items in this state are hidden from everyone except the owner and managers, and only the managers can edit them. Using Smart Folders, managers can quickly view all items that should be taken off the site. Ideally, a script would be developed to save these items to HTML or PDF, and then remove them from the system.  
• The ZODB saves past versions of every content item throughout the system, until unused versions are purged in a process called packing.  Out of the box, this task must be executed manually within the control panel of the ZMI.  If a ZODB that contains a highly dynamic Plone site is not packed regularly, the ZODB will get very large, require significantly more memory than the active content it contains, and performance will degrade.
• For automated packing and archiving stale content, a third-party product called Plone Maintenance might be useful, but we did not have time to evaluate it.
• We have a lot of process work-arounds that we haven't implemented in code.  In other words, we have defined standard practices, but the system doesn't enforce, or even necessarily encourage them.  It would not be impossible to implement all these processes, but it would require a significant, ongoing investment to keep a Plone instance in sync with dynamic organizational and project needs.

Third-Party Tools

• When using third-party products for Plone, to say that "Some Assembly is Required" would be an understatement. There is an active and enthusiastic community of Plone users, and many will unhesitatingly suggest products, whether they've used them or not, whether the product actually works, and whether it would be appropriate for the context.  This problem is not specific to the Plone world. Be skeptical, and always evaluate thoroughly.
• After researching dozens of products, and evaluating of 15-20 individual products, we are only using one third-party product on our production site, and even that product has problems. There were real benefits to be gained from it, and the problems we found were reproducible and easily understood, so we developed workarounds for them and deployed anyway.
• Instability, poor documentation, unfixed bugs, products that broke core Plone features, concerns about future migrations and ongoing maintenance, and complexity were among the reasons for ruling out most of the products that we tested.
  
• In one test instance, after installing a handful of products, I discovered that each product was adding its own roles, and there were now a total of sixteen roles in the ZMI. This may be a 'clean' way to handle product-specific permissions from the perspective of the product developer. In practice, it shifts the complexity burden to each Plone administrator by making it more difficult to manage user and group permissions. A more complex system is more likely to have vulnerabilities.
• Here is a quick example of our experience evaluating third-party products for a specific, unmet need:
  

Storing files like PDFs and PowerPoint files on the file system instead of in the ZODB is a widely-acknowledged best practice.  However, the 'file' content type included in Plone 2.1, doesn't follow this practice, and stores all uploaded files in the ZODB, which will degrade performance significantly over time.  There is also no way to limit the size of uploads, to check for viruses on upload, to manage file storage, etc.

In our survey of third-party products that would meet this need, the results were disappointing. One product was always broken in the ZMI, others had significant dependencies that would also need to be evaluated separately, some were framework products that would have required a great deal of custom code - an investment we were unwilling to make without assurance that it actually worked in the first place - and another would not appear in the products panel in Plone, even though it looked fine in the ZMI.

To be clear, I'm not saying that there aren't any products that will successfully meet this need, but that in the time we had allocated for evaluation and testing, none of the six we evaluated seemed promising enough to warrant additional time for testing, let alone deployment. There may well be a solution now, but we have no time left to return to it. Given non-infinite resources and patience, testing must end at some point, and we had to move on.

Workflow, Permissions and Security

• In phase two, we significantly customized Plone's workflow system to match our needs. Rather than describe the details here, please refer to the documents in the 'Our Workflows' folder, including:
• Explaining Our Workflow
• HumaniNet's Content Workflow (PDF, 95k)
• Workflow FAQ
• Potential Workflow Enhancements
  
• Although we actually have three distinct workflows in use, they were each designed to use the same adjectives (state) and transitions (verbs) to minimize user confusion.  In other words, some workflows are subsets of others, but the language is the same throughout the system. Our workflow is based on roles added by GroupSpaces.
• Although Plone can be configured to allow completely different workflows for different content types, and 'placeful' workflows are on the horizon, in my experience, workflow is one of the most difficult aspects of the systems for users to grasp, so reducing options and permutations should be a goal for the sake of usability.
• All of HumaniNet's information about people is stored in SalesForce.com, and we had hoped to create a near-real-time two-way connection between our Plone installation and our SalesForce.com database to share subscriber data. While there is a possibility that an OpenLDAP server could be used as a sort of proxy for this, or that a product could be designed to interact with the SalesForce API, preliminary research indicated that such an approach would be well beyond the scope of this project. For now, we have multiple-entry manual processes in place.
• The SalesForce.com/foundation recently awarded a grant to ONE/Northwest and Enfold Systems to build a connection between Plone and SalesForce.com based on PlonePAS, and this will likely be the best option once it is released later this year.