Update README.md with latest changes

README.md

@@ -1,6 +1,6 @@
 # rhedyn
 
-a quick and dirty static site generator.
+a quick and dirty static site generator with intelligent caching.
 
 Run this script from a directory containing markdown, templates and styles and a `dist/` directory will be created
 containing the rendered-out static assets ready for deployment.
@@ -10,22 +10,21 @@ directory where you run the tool.
 
 "Rhedyn" is Welsh for "fern", and is pronounced a bit like "read in". 
 
-
 ## Installation and usage
 
 The default config will look for `.md` files in the `markdown/` directory, `.scss` files in the `styles/` directory, and
-handlebars (`.hbs`) files in the `templates/` directory. 
+handlebars (`.hbs`) files in the `templates/` directory. It also processes images, SVG icons, static files, and generates favicons.
 
 The styles will be compiled using `sass` and output to a matching path in the `dist/` directory with a `.css` file
 extension. These file paths are made available to the template renderer, for inclusion as whatever tags you see fit.
 
 Markdown is compiled using `marked` with the `marked-code-preview` extension enabled, then passed to the template
-renderer in the `content` property.
+renderer in the `content` property. Images referenced in markdown are automatically processed and converted to responsive
+srcsets if matching images are found in the images task.
 
-Once the markdown content and template have been rendered out, the resulting `.html` file is output to `dist/`, again
+Once the markdown content and template have been rendered out, the resulting `.html` file is minified and output to `dist/`, 
 with the matching path (e.g. `markdown/recipes/soup.md` would be rendered to `dist/recipes/soup.html`).
 
-
 Usual stuff applies - install as a module locally or globally and call the default function:
 
 `npx rhedyn`
@@ -39,6 +38,16 @@ Usual stuff applies - install as a module locally or globally and call the defau
 You can also build a self-contained executable if you like chunky binaries with `npm run build`. That'll output to the
 `dist/` directory, and you can then put that binary wherever you like (probably somewhere in your $path).
 
+## Caching
+
+Rhedyn includes intelligent caching that tracks both file dependencies and configuration state. Tasks are only re-run when:
+- Input files have changed (detected via file hashes)
+- Configuration has changed (detected via state hashes)
+- Output files are missing (can be skipped by setting `opts.ignoreExisting` to `true`)
+
+Cache files are stored in `.cache/` by default and can be disabled by setting `cacheDir: false` in your config.
+
+If output file checks are skipped with `ignoreExisting`, only files that have changed inputs will be output.
 
 ## Configuration
 
@@ -52,65 +61,163 @@ If you want to extend the default config, it can be imported as `defaultConfig`
 #### opts
 
 The `opts` object in your config is for anything related to the project as a whole, rather than individual tasks. That
-means things like the base output directory, where to find templates for the renderer, etc. It'll be passed to the
+means things like the base output directory, cache directory, site metadata, etc. It'll be passed to the
 processor function as a key on the `meta` object.
 
-`opts` can also have an `include` property, an object containing task keys (e.g. "styles") and paths to include for
-that task, such as additional styles or templates. The path can be anywhere, not just local to the project - I use it to
+`opts` can also have an `include` property, an object containing task keys (e.g. "styles") and additional glob patterns to include for
+that task. The path can be anywhere, not just local to the project - I use it to
 include basic typography styles that I know I'll want in every project, but it could just as easily be used to produce a
 sort-of "theme" that could be shared across multiple projects.
 
-  ```
-    opts: {
-      baseDir: 'dist/',
-      templatesDirs: ['templates/'],
-      defaultTemplate: 'default',
-      include: { styles: ['~/.rhedyn/styles/'] }
-    },
-  ```
+```javascript
+opts: {
+  outDir: 'dist/',
+  runDir: process.cwd(),
+  cacheDir: '.cache',
+  clean: true,
+  ignoreExisting: false,
+  logLevel: 'debug',
+  include: { 
+    styles: [{ pattern: '~/.rhedyn/styles/*.scss' }] 
+  },
+  site: {
+    name: "My Website",
+    shortName: "My Site",
+    description: "A website generated from files using Rhedyn",
+    author: "Your Name",
+    url: "https://example.com",
+    language: "en-GB",
+    backgroundColor: "#ffffff",
+    themeColor: "#000000"
+  }
+}
+```
 
 #### tasks
 
-Tasks should be an array of objects that look something like this:
+Tasks can be either individual task objects or arrays of task objects that run in parallel. Tasks in different array elements run sequentially, allowing you to control dependencies between task groups.
+
+Example structure:
+```javascript
+tasks: [
+  [
+    // These tasks run in parallel
+    { name: "styles", ... },
+    { name: "icons", ... },
+    { name: "images", ... }
+  ],
+  // This task runs after the above group completes
+  { name: "pages", ... }
+]
+```
 
-  ```
-  {
-    name: "styles",
-    inputDirs: ["styles/"],
-    outputDir: "static/styles/",
-    inputFileExtension: ".scss",
-    outputFileExtension: ".css",
-    processor: compileSass
-  }
-  ```
-
-All properties are mandatory, and simply specify where we're reading the files in from and their extensions then where
-to put them once we've processed them. `inputDirs` is an array of paths, so you could include multiple paths here if you
-wish (some SCSS from a node module, for example).
-
-`processor` is a function that takes a file path as it's first argument, which is
-the file path to process, and `meta` as the second argument. `meta` contains the `opts` object, along with the
-already-processed filepaths from other tasks:
-
-  ```
-  {
-    opts: {
-      baseDir: 'dist/',
-      templateDirs: ['templates/'],
-      defaultTemplate: 'default',
-      include: { styles: ['~/.rhedyn/styles/'] }
+Each task object should look something like this:
+
+```javascript
+{
+  name: "styles",
+  inputFiles: [{ pattern: "styles/**/*.scss", ignore: "**/_*.scss" }],
+  stripPaths: ["styles/"],
+  outputDir: "static/styles/",
+  outputFileExtension: ".css",
+  processor: compileSass
+}
+```
+
+**Task Properties:**
+- `name`: Task identifier (required)
+- `inputFiles`: Array of glob pattern objects with `pattern` and optional `ignore` properties - if set, the task will be
+expanded for every file found
+- `stripPaths`: Array of path prefixes to remove from input paths when generating output paths
+- `outputDir`: Directory within `outDir` where processed files should be placed
+- `outputFileExtension`: File extension for processed files
+- `processor`: Function that processes the files (required)
+- Additional processor-specific properties (e.g., `imageSizes`, `quality` for image processing)
+
+**Input File Patterns:**
+```javascript
+inputFiles: [
+  { pattern: "styles/**/*.scss", ignore: "**/_*.scss" },
+  { pattern: "images/*.jpg" },
+  { pattern: "static/*" }
+]
+```
+
+#### processors
+
+A processor is a function that receives an object with `config` and `meta` properties and returns an object describing what was processed.
+
+The processor function signature:
+```javascript
+async function myProcessor({ config, meta }) {
+  // config contains the task configuration plus file-specific properties:
+  // - filePath: path to the input file being processed
+  // - fileOutputPath: calculated output path
+  // - fileOutputDir: directory for the output file
+  
+  // meta contains:
+  // - opts: global configuration
+  // - resources: results from previously completed tasks
+  
+  // Process the file...
+  
+  return {
+    detail: {}, // Optional: any metadata about the processed file
+    paths: [outputPath], // Array of output file paths (relative to outDir)
+    deps: { // Optional: dependencies for caching
+      paths: [inputFile1, inputFile2], // File dependencies
+      state: [] // State dependencies (tracked automatically)
     },
-    resources: { styles: [ 
-      { path: 'static/styles/main.css', detail: {} } 
-    ] }
+    ref: "unique-identifier" // Optional: reference key for this result
   }
-  ```
- 
-#### processors
+}
+```
+
+**Built-in Processors:**
+
+The following processors are available from `processors`:
+- `compileSass`: Compiles SCSS files to compressed CSS
+- `renderMarkdownWithTemplate`: Renders markdown with Handlebars templates, includes frontmatter support
+- `optimiseSvg`: Optimizes SVG files using SVGO
+- `copy`: Copies files without processing
+- `imageToWebP`: Converts images to WebP with multiple sizes for responsive images
+- `generateFavicons`: Generates favicon sets and web app manifests
+
+**Resources and Cross-Task References:**
+
+Processed files are made available to subsequent tasks via `meta.resources[taskName][ref]`. For example, the image processor makes processed images available to the markdown renderer for automatic srcset generation.
+
+Some examples can be found in `src/processors.js`, and you can find utility functions exported from this package as
+`utils`. The sample processors are also exported from this module as `processors`.
 
-A processor is just a function that takes a filepath and meta, returns an object with the processed result and
-optionally `detail`. `detail` can contain any properties you like, and is generally for annotating things like "date" or
-"author" that might be extracted from the source file (think frontmatter and the like).
+## Default Tasks
+
+The default configuration includes:
+
+1. **Parallel processing group:**
+   - `styles`: Compiles SCSS files to CSS
+   - `icons`: Optimizes SVG icons
+   - `images`: Converts JPG images to WebP with multiple sizes
+   - `static files`: Copies static files
+   - `favicons`: Generates favicon sets from source images
+
+2. **Sequential processing:**
+   - `pages`: Renders Markdown files with Handlebars templates (runs after the parallel group to access processed resources)
+
+## Logging
+
+Rhedyn includes comprehensive logging with configurable levels:
+- `silent` (1): No output
+- `error` (2): Errors only  
+- `warn` (3): Warnings and errors
+- `info` (4): General information (default)
+- `debug`: Detailed debugging information
+- `trace` (6): Maximum verbosity
+
+Set the log level in your config:
+```javascript
+opts: {
+  logLevel: 'debug'
+}
+```
 
-Some examples can be found in `src/processors.js`, and you can find some utility functions exported from this package as
-`utils`. The sample processors are also exported from this module, as `processors`.