Using Perforce Chronicle for application configuration

Posted on November 7, 2012 by Logan McGrath

Tags: perforce, configuration management

Estimated reading time: 43m 11s

Following Paul Hammant’s post App-config workflow using SCM and subsequent proof of concept backed by Git, I will show that an app-config application backed by Perforce is possible using Perforce Chronicle.

Perforce and permissions for branches

Perforce is an enterprise-class source control management (SCM) system, remarkably similar to Subversion (Subversion was inspired by Perforce :) Perforce is more bulletproof than Subversion in many ways and it’s generally faster. Git does not impose any security constraints or permissions on branches, Perforce gives comprehensive security options allowing you to control access to different branches: for example, development, staging, and production. Subversion, however, can support permissions on branches with some extra configuration (Apache plus mod_dav_svn/mod_dav_authz). For these reasons, Perforce is a better option for storing configuration data than either Git or Subversion.

Perforce CMS as an application server

Perforce Chronicle is a content management system (CMS) using Perforce as the back-end store for configuration and content. The app-config application is built on top of Chronicle because Perforce does not offer a web view into the depot the way Subversion can through Apache. Branching and maintaining divergence between environments can be managed through the user interface, and Chronicle provides user authentication and management, so access between different configuration files can be restricted appropriately. The INSTALL.txt file that is distributed with Chronicle helps with an easy install, mine being set up to run locally from http://localhost.

There is a key issue in using Chronicle, however. The system is designed for the management of content and not necessarily arbitrary files. In order to make the app-config application work, I had to add a custom content type and write a module. Configuration and HTML are both plain-text content, so I created a ” Plain Text” content type with the fields title and content:

Go to “Manage” > “Content Types”
Click “Add Content Type”
Enter the following information:

Id:       plaintext
Label:    Plain Text
Group:    Assets
Elements:

[title]
type = text
options.label = Title
options.required = true
display.tagName = h1
display.filters.0 = HtmlSpecialChars

[content]
type = textarea
options.label = "Content"
options.required = true
display.tagName = pre
display.filters.0 = HtmlSpecialChars

Click “Save”.

The Config App

I’ve borrowed heavily from Paul’s app-config HTML page, which uses AngularJS to manage the UI and interaction with the server. Where Paul’s app-config app used the jshon command to encode and decode JSON, Zend Framework has a utility class for encoding, decoding, and pretty-printing JSON, and Chronicle also ships with the simplediff utility for performing diffs with PHP.

The source JSON configuration is the same, albeit sorted:

{
  "bannedNicks": [
    "derek",
    "dino",
    "ffff",
    "jjjj",
    "werwer"
  ],
  "defaultErrorReciever": "piglet@thoughtworks.com",
  "lighton": true,
  "loadMaxPercent": "88",
  "nextShutdownDate": "8\/9\/2012"
}

The index.html page has been modified from the original to support only the basic commit and diffs functionality:

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
        "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html lang="en" xmlns:ng="http://angularjs.org">
<head>
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
  <title>Configuration application (alpha)</title>
  <script type="text/javascript" ng:autobind src="http://code.angularjs.org/0.9.19/angular-0.9.19.min.js"></script>
  <style type="text/css">
    ins { color: #00CC00; text-decoration: none; }
    del { color: #CC0000; text-decoration: none; }
  </style>
</head>
<body ng:controller="AppCfg">
<script type="text/javascript">
    function AppCfg($resource, $xhr) {
        var self = this;
        this.newNickname = "";
        this.svrMessage;
        this.message;
        this.cfg = $resource("/appconfig/stack_configuration.json").get({});

        this.save = function() {
            self.cfg.$save({message: self.message}, function() {
                alert("Config saved to server");
            }, function() {
                alert("ERROR on save");
            });
            self.message = "";
        };

        this.newNick = function() {
            self.cfg.bannedNicks.push(self.newNickname);
            self.newNickname = "";
        };

        this.diffs = function() {
            $xhr("post", "/appconfig/diffs/stack_configuration.json", angular.toJson(self.cfg), function(code, svrMessage) {
                self.svrMessage = svrMessage;
            });
        };

        this.deleteNick = function(nick) {
            var oldBannedNicks = self.cfg.bannedNicks;
            self.cfg.bannedNicks = [];
            angular.forEach(oldBannedNicks, function(n) {
                if (nick != n) {
                    self.cfg.bannedNicks.push(n);
                }
            });
        };
    }

    AppCfg.$inject = ["$resource", "$xhr"];
</script>
  Light is on:  <input type="checkbox" name="cfg.lighton"/> <br/>
  Default Error Reciever (email): <input name="cfg.defaultErrorReciever" ng:validate="email"/> <br/>
  Max Load Percentage: <input name="cfg.loadMaxPercent" ng:validate="number:0:100"/> <br/>
  Next Shutdown Date: <input name="cfg.nextShutdownDate" ng:validate="date"/> <br/>
  Banned nicks:
      <ol>
        <li ng:repeat="nick in cfg.bannedNicks"><span>{{nick}} &nbsp;&nbsp;<a ng:click="deleteNick(nick)">[X]</a></span></li>
    </ol>
  <form ng:submit="newNick()">
    <input type="text" name="newNickname" size="20"/>
    <input type="submit" value="&lt;-- Add Nick"/><br/>
  </form>
  <hr/>
  <button ng:click="diffs()">View Diffs</button><br/>
  <button ng:disabled="{{!message}}" ng:click="save()">Commit Changes</button> Commit Message: <input name="message"></button><br/>
  Last Server operation: <br/>
  <div ng:bind="svrMessage | html:'unsafe'">
  </div>
</body>
</html>

Both of these assets were added by performing:

Click “Add” from the top navbar
Click “Add Content”
Select “Assets” > “Plain Text”
For “Title”, enter “index.html” or “stack_configuration.json”
Paste in the appropriate “Content”
Click “URL”, select “Custom”, and enter the same value as “Title” (otherwise, Chronicle will convert underscores to dashes, so be careful!)
Click “Save”, enter a commit message, then click the next “Save”
Both assets should be viewable as mangled Chronicle content entries from http://localhost/index.html and http://localhost/stack_configuration.json. You normally will not use these URLs.

At this point, neither asset is actually usable. Most content is heavily decorated with additional HTML and then displayed within a layout template, but I want both the index.html and stack_configuration.json assets to be viewable as standalone files and provide a REST interface for AngularJS to work against.

Come back PHP! All is forgiven

Chronicle is largely built using Zend Framework and makes adding extra modules to the system pretty easy. My module needs to be able to display plaintext assets, update their content using an HTTP POST, and provide diffs between the last commit and the current content.

To create the module, the following paths need to be added:

INSTALL/application/appconfig
INSTALL/application/appconfig/controllers
INSTALL/application/appconfig/views/scripts/index

Declare the module with INSTALL/application/appconfig/module.ini:

version = 1.0
description = Application config proof of concept
icon = images/icon.png
tags = config

[maintainer]
name = Perforce Software
email = support@perforce.com
url = http://www.perforce.com

[routes]
appconfig.type = Zend_Controller_Router_Route_Regex
appconfig.route = 'appconfig/(.+)'
appconfig.reverse = appconfig/%s
appconfig.defaults.module = appconfig
appconfig.defaults.controller = index
appconfig.defaults.action = index
appconfig.map.resource = 1

appconfig-operation.type = Zend_Controller_Router_Route_Regex
appconfig-operation.route = 'appconfig/([^/]+)/(.+)'
appconfig-operation.reverse = appconfig/%s/%s
appconfig-operation.defaults.module = appconfig
appconfig-operation.defaults.controller = index
appconfig-operation.defaults.action = index
appconfig-operation.map.action = 1
appconfig-operation.map.resource = 2

Add a view script for displaying plaintext assets, INSTALL/application/appconfig/views/scripts/index/index.phtml:

<?=$this->entry->getValue('content') ?>

Add a view script for displaying diffs, INSTALL/application/appconfig/views/scripts/index/diffs.phtml:

<pre><?=$this->diffs ?></pre>

And a controller at INSTALL/application/appconfig/controllers/IndexController.phtml:

<?php

defined('LIBRARY_PATH') or define('LIBRARY_PATH', dirname(__DIR__));
require_once LIBRARY_PATH . '/simplediff/simplediff.php';

class Appconfig_IndexController extends Zend_Controller_Action
{
    private $entry;
    
    private $mimeTypes = array(
        '.html' => 'text/html',
        '.json' => 'application/json',
    );
    
    public function preDispatch()
    {
        $request = $this->getRequest();
        $request->setParams(Url_Model_Url::fetch($request->getParam('resource'))->getParams());
        $this->entry = P4Cms_Content::fetch($request->getParam('id'), array('includeDeleted' => true));
    }
    
    public function indexAction()
    {
        $this->getResponse()->setHeader('Content-Type', $this->getMimeType(), true);
        $this->view->entry = $this->entry;
        
        if ($this->getRequest()->isPost()) {
            $this->entry->setValue('content', $this->getJsonPost());
            $this->entry->save($this->getRequest()->getParam('message'));
        }
    }
    
    private function getMimeType()
    {
        $url = $this->entry->getValue('url');
        $suffix = substr($url['path'], strrpos($url['path'], '.'));
        
        if (array_key_exists($suffix, $this->mimeTypes)) {
            return $this->mimeTypes[$suffix];
        } else {
            return 'text/plain';
        }
    }
    
    public function diffsAction()
    {
        $this->getResponse()->setHeader('Content-Type', 'text/html', true);
        $this->view->diffs = htmlDiff($this->entry->getValue('content'), $this->getJsonPost());
    }
    
    public function postDispatch()
    {
        $this->getHelper('layout')->disableLayout();
    }
    
    private function getJsonPost()
    {
        if ($this->getRequest()->isPost()) {
            return $this->prettyPrint(file_get_contents('php://input'));
        } else {
            throw new Exception('Can\'t get JSON without POST');
        }
    }
    
    private function prettyPrint($json)
    {
        $array = Zend_Json::decode($json);
        $this->sort($array);
        
        return Zend_Json::prettyPrint(Zend_Json::encode($array), array('indent' => ' '));
    }
    
    private function sort(array &$array)
    {
        if (count(array_filter(array_keys($array), 'is_string')) > 0) {
            ksort($array);
        }
        
        foreach($array as &$value) {
            if (is_array($value)) {
                $this->sort($value);
            }
        }
    }
}

AngularJS

After all files are in place, Chronicle needs to be notified that the new module exists by going to “Manage” > “Modules”, where the “Appconfig” module will be listed if all goes well :) Both assets will now be viewable from http://localhost/appconfig/index.html and http://localhost/appconfig/stack_configuration.json. AngularJS’ resource service is used in index.html to fetch stack_configuration.json and post changes back.

From http://localhost/appconfig/index.html, the data from stack_configuration.json is loaded into the form:

Edits to stack_configuration.json can be made using the form, and the diffs viewed by clicking on “View Diffs”:

The changes can be saved by entering a commit message and clicking “Commit Changes”. After which, clicking “View Diffs” will show no changes:

To show that edits have in fact been made to stack_configuration.json, go to http://localhost/stack_configuration.json, select “History” and click on ” History List”:

Chronicle also provides an interface for viewing diffs between revisions:

Disk Usage

Something to remember in using Chronicle is that each resource requested from Perforce is written to disk before being served to the client. This means that for each request to index.html, Chronicle allocates a new Perforce workspace, checks out the associated file, serves it to the client, then deletes the file and the workspace at the end of the request. This allocate/checkout/serve/delete cycle executes for stack_configuration.json and every other resource in the system.

@TODO

Security!

There’s one major flaw with the appconfig module: it performs zero access checks. By default, Chronicle can be configured to disallow anonymous access by going to “Manage” > “Permissions” and deselecting all permissions for ” anonymous” and “members”. Logging out and attempting to access either http://localhost/appconfig/stack_configuration.json or http://localhost/appconfig/index.html will now give an error page and prompt you to log in. Clicking “New User” will also give an error, as anonymous users don’t have the permission to create users.

Access rights on content are checked by the content module, but are also hard-coded in the associated controllers as IF-statements. A better solution will be required for proper access management in the appconfig module.

Better integration

Chronicle’s content module provides JSON integration for most of its actions, but these mostly exist to support the Dojo Toolkit-enabled front-end. Integrating with these actions over JSON requires detailed knowledge of Chronicle’s form structures.

Chronicle has some nice interfaces for viewing diffs. If I could call those up from index.html I would be major happy :)

Automatic creation of plaintext content type

Before the appconfig module is usable, the plaintext content type has to be created. I would like to automate creation of the plaintext content type when the module is first enabled.

Making applications aware of updates to configuration

When stack_configuration.json is updated, there’s no way to notify applications to the change, and no interface provided so they may poll for changes. I’m not entirely sure at this point what an appropriate solution would look like. In order to complete the concept, I’d first have to create a client app dependent on that configuration.

Better interfaces for manipulating plaintext assets

I had to fiddle with index.html quite a bit. This basically involved editing a local copy of index.html, then pasting the entire contents into the associated form in Chronicle. I have not tried checking out index.html directly from Perforce, and I imagine that any edits would need to be made within Chronicle. GitHub offers an in-browser raw editor, and something like that would be real handy in Chronicle.

Handling conflicts

There is no logic in the appconfig module to catch conflicts if there are two users editing the same file. Conflicts are detectable because an exception is thrown if there is a conflict, but I’m not sure what the workflow for resolution is in Chronicle terms, or how to integrate with it. Who wins?

Working with branches

I did not take the time to see how Chronicle manages branches. I will need to verify that Chronicle and the appconfig module can work with development, staging, and production branches, with maintained divergence. For example, we’re still trying to figure out how to attach visual clients like P4V to the repository and work independently of Chronicle.

Kudos

I would like to thank the guys at Perforce for their assistance and answering all my questions as I worked with Chronicle, especially Randy Defauw.

Update 06/27/2021

This is the first blog post I have ever written. Paul Hammant, who I had met in other contexts previously, happened to be working out of the ThoughtWorks office in Dallas, TX the very same day I started at ThoughtWorks. He asked me if I knew PHP, which I did, and set me off to explore Perforce Chronicle as a solution for managing configuration.

I had never written professionally before or been aware of configuration management: I was very lucky to explore a passion space that Paul has worked within for a very long time. I don’t believe I ever gave him a proper thanks. He gave me an opportunity that probably not a lot of people get in their early careers, and it was an invaluable experience that I learned a lot from and think about fairly often.

The other posts in this series were also written with guidance from Paul:

The subject of configuration as described in these posts is still fresh even after nearly ten years. Even now configuration as code still doesn’t have a perfect solution, though products have become available that make managing configuration easier. Changing configuration in a running process as a general solution remains elusive, as supporting it imposes a lot of constraints on design.

On a more personal note: today is Pride. This is the first Pride I’ve ever participated in and only in the last two years have I felt safe enough to come out in circles beyond close friends. I was out to Paul but only as a detail I confided in passing. When I was working with Paul on these posts he advised that I should consider relocating to the Bay Area. In January, 2013, I moved to San Francisco. Discovering my own life as a person was set by Paul being brave enough to share a deeply personal piece of advice. I’m so thankful he said it, and I’m glad I listened.

Please share with me your thoughts and feedback!