Appender and Encoder

Scoopi uses appender to append data to output and encoder to encode or alter the output format. At present, it comes with two appender - FileAppender and ListAppender.

FileAppender

So far, examples use FileAppender of default steps defined in steps-default.yml which is packaged with scoopi distribution jar. The appender snippet is as below.

appender:
  class: "org.codetab.scoopi.step.load.DataAppender"
  previous: filter
  next: end
  plugins: [
    plugin: { 
      name: dataFile, 
      class: "org.codetab.scoopi.plugin.appender.FileAppender",
      file: "output/data.txt", 
      plugins: [ 
         plugin: { 
           name: csv,
           delimiter: "|",
           includeTags: false,               
           class: "org.codetab.scoopi.plugin.encoder.CsvEncoder"
         } 
      ]
    }          
  ]

We can override the step in task level or create altogether new steps as detailed in WorkFlow, Steps and Plugins.

To use appender, first define a step that uses class DataAppender which is responsible for unique appender creation common to many task. The appender itself is a plugin such as FileAppender, List appender etc., and it defined in plugins array. For example, say we want two output files with different delimiters then override the step as follows

appender:
  class: "org.codetab.scoopi.step.load.DataAppender"
  previous: filter
  next: end
  plugins: [
    plugin: {
      name: file1,
      class: "org.codetab.scoopi.plugin.appender.FileAppender",
      file: "output/data.csv",
      plugins: [
         plugin: {
           name: csv,
           delimiter: ",",
           class: "org.codetab.scoopi.plugin.encoder.CsvEncoder"
         }
      ]
    },
    plugin: {
      name: file2,
      class: "org.codetab.scoopi.plugin.appender.FileAppender",
      file: "output/data.dump",
      plugins: [
         plugin: {
           name: pipe,
           delimiter: "|",
           class: "org.codetab.scoopi.plugin.encoder.CsvEncoder"
         }
      ]
    }
  ]

In next chapter, we show how to handle JavaScript and Ajax driven pages using Selenium WebDriver.