PyPerforce: Tutorial: Custom handling of Perforce output

PyPerforce provides a powerful tool for customising processing of Perforce command output by letting you write your own output handler objects that support the IOutputConsumer interface.

This can be useful in situations where you want to process results as they come in rather than waiting until the entire command is complete. For example, a p4 sync command can take quite a while to finish and so you may want to display progress of the command to the user while it is running.

Using custom output handling is almost a necessity for writing scalable applications that may be processing massive amounts of data or printing the contents of large files from the repository that may not fit in system memory.

The IOutputConsumer interface

The IOutputConsumer interface consists of several methods, each of which are called during the execution of a command to handle output of a particular type of data.

import protocols

class IOutputConsumer(protocols.Interface):

  # Called when an info, warning or error message is output (eg from 'p4 sync')
  # Passed a perforce.Message object that has isInfo(), isWarning() and
  # isError() methods that can be used to determine the message type.
  # Use str(message) to obtain the message text.
  def outputMessage(message):
    pass
  
  # Called when a single data record is output (eg from 'p4 fstat')
  # Passed a Python dictionary that has a key-value pair for each record field.
  def outputRecord(record):
    pass
  
  # Called when a form is output (eg from 'p4 client -o')
  # Passed a perforce.forms.Form object that contains the parsed form data
  def outputForm(form):
    pass
  
  # Called when a chunk of binary file data is output
  # Passed a Python string containing the byte data.
  def outputBinary(data):
    pass
    
  # Called when a chunk of text file data is output
  # Passed a Python string containing the byte data.
  def outputText(data):
    pass
    
  # Called when the command finishes executing
  def finished():
    pass

Writing your own consumer class

To implement your own custom output handling you need to write a class that provides the IOutputConsumer interface.

In this example we are going to implement custom handling for the p4 sync command to print out the synced files as they go and provide a summary of any errors and warnings at the end of the operation.

from protocols import advise
from perforce.results import IOutputConsumer

class SyncHandler(object):

  # Flag this class as implementing IOutputConsumer
  advise(instancesProvide=[IOutputConsumer])
  
  def __init__(self):
    self.errors = []
    self.warnings = []
  
  # A message is output for each file that is synced and for each
  # error or warning that was issued.
  def outputMessage(message):
    # Output info messages promptly and save up errors and warnings
    if message.isInfo():
      print str(message)
    elif message.isWarning():
      self.warnings.append(str(message))
    elif message.isError():
      self.errors.append(str(message))
     
  # Called once the sync operation is complete 
  def finish(self):
    # Output any warnings and errors that occurred as well as a summary
    for msg in self.warnings:
      print "Warning:", msg
    for msg in self.errors:
      print "Error:", msg
    print "%i warnings, %i errors" % (len(self.warnings), len(self.errors))
      
  # Helper method for ignoring output we're not interested in
  def _doNothing(self, *args, **kw):
    pass
    
  outputForm = _doNothing
  outputText = _doNothing
  outputBinary = _doNothing
  outputRecord = _doNothing

Using the consumer class

Using the new custom output handling class when running a command is done by passing an instance of the class to the output parameter of the Connection.run() method.

The object passed to the output parameter is returned from the Connection.run() method.

For example:

from perforce import Connection
p4 = Connection()
p4.connect()
try:
  p4.run('sync', output=SyncHandler())
finally:
  p4.disconnect()