Mirrors/Stirling-PDF

Fork 0

mirror of https://github.com/Frooodle/Stirling-PDF.git synced 2025-12-18 20:04:17 +01:00

Anthony Stirling e32a0c04c4 docs

2025-10-09 18:20:22 +01:00

20 KiB

Raw Permalink Blame History

Stirling-PDF Developer Guide

1. Introduction

Stirling-PDF is a robust, locally hosted, web-based PDF manipulation tool. This guide focuses on Docker-based development and testing, which is the recommended approach for working with the full version of Stirling-PDF.

2. Project Overview

Stirling-PDF is built using:

Spring Boot (Backend API)
React + TypeScript + Vite (Frontend V2)
Mantine UI + TailwindCSS (UI Framework)
PDFBox (PDF manipulation)
LibreOffice (Document conversion)
qpdf (PDF processing)
PDF.js (Client-side PDF rendering)
Embedded-PDF (PDF viewer component)
Docker
Lombok

3. Development Environment Setup

Prerequisites

Docker
Git
Java JDK 17 or later (JDK 21 recommended)
Gradle 7.0 or later (Included within the repo)
Node.js 18+ and npm (for frontend development)

Setup Steps

Clone the repository:

git clone https://github.com/Stirling-Tools/Stirling-PDF.git
cd Stirling-PDF

Install Docker and JDK17 if not already installed.
Install a recommended IDE:
- VSCode (recommended for frontend)
  1. Open VS Code.
  2. When prompted, install the recommended extensions.
  3. Alternatively, open the command palette (Ctrl + Shift + P or Cmd + Shift + P on macOS) and run:
```
Extensions: Show Recommended Extensions
```
  1. Install the required extensions from the list.
- IntelliJ IDEA (recommended for backend)
- Eclipse (alternative for backend)
Lombok Setup Stirling-PDF uses Lombok to reduce boilerplate code. Some IDEs, like Eclipse, don't support Lombok out of the box. To set up Lombok in your development environment: Visit the Lombok website for installation instructions specific to your IDE.
Add environment variable For local testing, you should generally be testing the full 'Security' version of Stirling PDF. To do this, you must add the environment flag DOCKER_ENABLE_SECURITY=true to your system and/or IDE build/run step.

4. Project Structure

Stirling-PDF/
├── .github/               # GitHub-specific files (workflows, issue templates)
├── configs/               # Configuration files used by stirling at runtime (generated at runtime)
├── cucumber/              # Cucumber test files
│   ├── features/
├── customFiles/           # Custom static files and templates (generated at runtime used to replace existing files)
├── docs/                  # Documentation files
├── exampleYmlFiles/       # Example YAML configuration files
├── frontend/              # React frontend application (V2)
│   ├── src/
│   │   ├── components/    # React components
│   │   ├── tools/         # PDF tool implementations
│   │   ├── contexts/      # React contexts (FileContext, etc.)
│   │   ├── hooks/         # Custom React hooks
│   │   ├── services/      # API and processing services
│   │   └── i18n.ts        # Internationalization config
│   ├── public/
│   │   └── locales/       # Translation JSON files
│   └── package.json
├── images/                # Image assets
├── pipeline/              # Pipeline-related files (generated at runtime)
├── scripts/               # Utility scripts
├── src/                   # Backend source code
│   ├── main/
│   │   ├── java/
│   │   │   └── stirling/
│   │   │       └── software/
│   │   │           └── SPDF/
│   │   │               ├── config/
│   │   │               ├── controller/
│   │   │               │   ├── api/    # REST API endpoints
│   │   │               │   └── web/    # Web controllers
│   │   │               ├── model/
│   │   │               ├── repository/
│   │   │               ├── service/
│   │   │               └── utils/
│   │   └── resources/
│   │       └── static/    # Legacy static assets
│   └── test/
│       └── java/
│           └── stirling/
│               └── software/
│                   └── SPDF/
├── build.gradle           # Gradle build configuration
├── Dockerfile             # Main Dockerfile
├── Dockerfile.ultra-lite  # Dockerfile for ultra-lite version
├── Dockerfile.fat         # Dockerfile for fat version
├── docker-compose.yml     # Docker Compose configuration
└── test.sh                # Test script to deploy all docker versions and run cuke tests

5. Docker-based Development

Stirling-PDF offers several Docker versions:

Full: All features included
Ultra-Lite: Basic PDF operations only
Fat: Includes additional libraries and fonts predownloaded

Example Docker Compose Files

Stirling-PDF provides several example Docker Compose files in the exampleYmlFiles directory, such as:

docker-compose-latest.yml: Latest version without login and security features
docker-compose-latest-security.yml: Latest version with login and security features enabled
docker-compose-latest-fat-security.yml: Fat version with login and security features enabled

These files provide pre-configured setups for different scenarios. For example, here's a snippet from docker-compose-latest-security.yml:

services:
  stirling-pdf:
    container_name: Stirling-PDF-Security
    image: docker.stirlingpdf.com/stirlingtools/stirling-pdf:latest
    deploy:
      resources:
        limits:
          memory: 4G
    healthcheck:
      test: ["CMD-SHELL", "curl -f http://localhost:8080/api/v1/info/status | grep -q 'UP' && curl -fL http://localhost:8080/ | grep -q 'Please sign in'"]
      interval: 5s
      timeout: 10s
      retries: 16
    ports:
      - "8080:8080"
    volumes:
      - ./stirling/latest/data:/usr/share/tessdata:rw
      - ./stirling/latest/config:/configs:rw
      - ./stirling/latest/logs:/logs:rw
    environment:
      DOCKER_ENABLE_SECURITY: "true"
      SECURITY_ENABLELOGIN: "true"
      PUID: 1002
      PGID: 1002
      UMASK: "022"
      SYSTEM_DEFAULTLOCALE: en-US
      UI_APPNAME: Stirling-PDF
      UI_HOMEDESCRIPTION: Demo site for Stirling-PDF Latest with Security
      UI_APPNAMENAVBAR: Stirling-PDF Latest
      SYSTEM_MAXFILESIZE: "100"
      METRICS_ENABLED: "true"
      SYSTEM_GOOGLEVISIBILITY: "true"
      SHOW_SURVEY: "true"
    restart: on-failure:5

To use these example files, copy the desired file to your project root and rename it to docker-compose.yml, or specify the file explicitly when running Docker Compose:

docker-compose -f exampleYmlFiles/docker-compose-latest-security.yml up

Building Docker Images

Stirling-PDF uses different Docker images for various configurations. The build process is controlled by environment variables and uses specific Dockerfile variants. Here's how to build the Docker images:

Set the security environment variable:

export DOCKER_ENABLE_SECURITY=true  # or false to disable login and security features for builds

Build the project with Gradle:
```
./gradlew clean build
```

Build the Docker images:

For the latest version:

docker build --no-cache --pull --build-arg VERSION_TAG=alpha -t stirlingtools/stirling-pdf:latest -f ./Dockerfile .

For the ultra-lite version:

docker build --no-cache --pull --build-arg VERSION_TAG=alpha -t stirlingtools/stirling-pdf:latest-ultra-lite -f ./Dockerfile.ultra-lite .

For the fat version (with login and security features enabled):

export DOCKER_ENABLE_SECURITY=true
docker build --no-cache --pull --build-arg VERSION_TAG=alpha -t stirlingtools/stirling-pdf:latest-fat -f ./Dockerfile.fat .

Note: The --no-cache and --pull flags ensure that the build process uses the latest base images and doesn't use cached layers, which is useful for testing and ensuring reproducible builds. however to improve build times these can often be removed depending on your usecase

6. Testing

Comprehensive Testing Script

Stirling-PDF provides a test.sh script in the root directory. This script builds all versions of Stirling-PDF, checks that each version works, and runs Cucumber tests. It's recommended to run this script before submitting a final pull request.

To run the test script:

./test.sh

This script performs the following actions:

Builds all Docker images (full, ultra-lite, fat).
Runs each version to ensure it starts correctly.
Executes Cucumber tests against the main version and ensures feature compatibility. In the event these tests fail, your PR will not be merged.

Note: The test.sh script will run automatically when you raise a PR. However, it's recommended to run it locally first to save resources and catch any issues early.

Full Testing with Docker

Build and run the Docker container per the above instructions
Access the application at http://localhost:8080 and manually test all features developed.

Local Testing (Frontend and Backend)

For quick iterations and development, you can run the frontend and backend separately:

Backend Development

Run the backend:
```
./gradlew bootRun
```
The backend API will be available at http://localhost:8080
API documentation is available at http://localhost:8080/swagger-ui/index.html

Frontend Development

Install dependencies (first time only):
```
cd frontend
npm install
```
Start the development server:
```
npm run dev
```
The frontend will be available at http://localhost:5173
Vite automatically proxies API calls from /api/* to the backend at localhost:8080

Important notes:

Frontend requires the backend to be running for full functionality
Hot module replacement (HMR) enables instant updates during development
Local testing doesn't include features that depend on external tools like qpdf, LibreOffice, or Python scripts.
Always verify your changes in the full Docker environment before submitting pull requests, as some integrations and features will only work in the complete setup.

7. Contributing

Fork the repository on GitHub.
Create a new branch for your feature or bug fix.
Make your changes and commit them with clear, descriptive messages and ensure any documentation is updated related to your changes.
Test your changes thoroughly in the Docker environment.
Run the test.sh script to ensure all versions build correctly and pass the Cucumber tests:
```
./test.sh
```
Push your changes to your fork.
Submit a pull request to the main repository.
See additional contributing guidelines.

When you raise a PR:

The test.sh script will run automatically against your PR.
The PR checks will verify versioning and dependency updates.
Documentation will be automatically updated for dependency changes.
Security issues will be checked using Snyk and PixeeBot.

Address any issues that arise from these checks before finalizing your pull request.

8. API Documentation

API documentation is available at /swagger-ui/index.html when running the application. You can also view the latest API documentation here.

9. Customization

Stirling-PDF can be customized through environment variables or a settings.yml file. Key customization options include:

Application name and branding
Security settings
UI customization
Endpoint management
Maximum DPI for PDF to image conversion (system.maxDPI)

When using Docker, pass environment variables using the -e flag or in your docker-compose.yml file.

Example:

docker run -p 8080:8080 -e APP_NAME="My PDF Tool" stirling-pdf:full

Refer to the main README for a full list of customization options.

10. Frontend Development (V2)

Architecture Overview

The V2 frontend is designed for stateful document processing:

Users upload PDFs once, then chain tools (split → merge → compress → view)
File state and processing results persist across tool switches
No file reloading between tools - performance critical for large PDFs (up to 100GB+)

Key Components

FileContext - Central State Management

Location: frontend/src/contexts/FileContext.tsx

Active files: Currently loaded PDFs and their variants
Tool navigation: Current mode (viewer/pageEditor/fileEditor/toolName)
Memory management: PDF document cleanup, blob URL lifecycle, Web Worker management
IndexedDB persistence: File storage with thumbnail caching
Preview system: Tools can preview results without context pollution

Critical: All file operations go through FileContext. Don't bypass with direct file handling.

Processing Services

enhancedPDFProcessingService: Background PDF parsing and manipulation
thumbnailGenerationService: Web Worker-based with main-thread fallback
fileStorage: IndexedDB with LRU cache management

Tool Development

Architecture: Modular hook-based system with clear separation of concerns:

useToolOperation (frontend/src/hooks/tools/shared/useToolOperation.ts): Main orchestrator hook
- Coordinates all tool operations with consistent interface
- Integrates with FileContext for operation tracking
- Handles validation, error handling, and UI state management
Supporting Hooks:
- useToolState: UI state management (loading, progress, error, files)
- useToolApiCalls: HTTP requests and file processing
- useToolResources: Blob URLs, thumbnails, ZIP downloads
Utilities:
- toolErrorHandler: Standardized error extraction and i18n support
- toolResponseProcessor: API response handling (single/zip/custom)
- toolOperationTracker: FileContext integration utilities

Three Tool Patterns:

Pattern 1: Single-File Tools (Individual processing)

Backend processes one file per API call
Set multiFileEndpoint: false
Examples: Compress, Rotate

return useToolOperation({
  operationType: 'compress',
  endpoint: '/api/v1/misc/compress-pdf',
  buildFormData: (params, file: File) => { /* single file */ },
  multiFileEndpoint: false,
});

Pattern 2: Multi-File Tools (Batch processing)

Backend accepts MultipartFile[] arrays in single API call
Set multiFileEndpoint: true
Examples: Split, Merge, Overlay

return useToolOperation({
  operationType: 'split',
  endpoint: '/api/v1/general/split-pages',
  buildFormData: (params, files: File[]) => { /* all files */ },
  multiFileEndpoint: true,
  filePrefix: 'split_',
});

Pattern 3: Complex Tools (Custom processing)

Tools with complex routing logic or non-standard processing
Provide customProcessor for full control
Examples: Convert, OCR

return useToolOperation({
  operationType: 'convert',
  customProcessor: async (params, files) => { /* custom logic */ },
});

Benefits:

No Timeouts: Operations run until completion (supports 100GB+ files)
Consistent: All tools follow same pattern and interface
Maintainable: Single responsibility hooks, easy to test and modify
i18n Ready: Built-in internationalization support
Type Safe: Full TypeScript support with generic interfaces
Memory Safe: Automatic resource cleanup and blob URL management

Adding a New Tool

See ADDING_TOOLS.md for a complete guide to creating new PDF tools.

Internationalization

Translations are stored in JSON files at frontend/public/locales/{language-code}/translation.json.

To use translations in React components:

import { useTranslation } from 'react-i18next';

function MyComponent() {
  const { t } = useTranslation();

  return (
    <div>
      <h1>{t('myTool.title')}</h1>
      <p>{t('myTool.description')}</p>
    </div>
  );
}

See HowToAddNewLanguage.md for details on adding new languages.

11. Backend Development

Adding a New API Endpoint

Create a New Controller:

Create a new Java class in the src/main/java/stirling/software/SPDF/controller/api directory.
Annotate the class with @RestController and @RequestMapping to define the API endpoint.
Ensure to add API documentation annotations like @Tag and @Operation.

package stirling.software.SPDF.controller.api;

import org.springframework.web.bind.annotation.*;
import org.springframework.web.multipart.MultipartFile;
import io.swagger.v3.oas.annotations.Operation;
import io.swagger.v3.oas.annotations.tags.Tag;

@RestController
@RequestMapping("/api/v1/pdf")
@Tag(name = "General", description = "General APIs")
public class NewFeatureController {

    @PostMapping("/new-feature")
    @Operation(summary = "New Feature", description = "This is a new feature endpoint. Input:PDF Output:PDF Type:SISO")
    public ResponseEntity<byte[]> newFeature(
        @RequestPart("fileInput") MultipartFile file,
        @RequestParam("param1") String param1) {

        // Process PDF
        byte[] result = processFile(file, param1);

        return ResponseEntity.ok()
            .header("Content-Disposition", "attachment; filename=output.pdf")
            .contentType(MediaType.APPLICATION_PDF)
            .body(result);
    }
}

Define the Service Layer: (Optional but recommended)

Create a new service class in the src/main/java/stirling/software/SPDF/service directory.
Implement the business logic for the new feature.

package stirling.software.SPDF.service;

import org.springframework.stereotype.Service;

@Service
public class NewFeatureService {

    public byte[] processFile(MultipartFile file, String param1) {
        // Implement business logic here
        return processedBytes;
    }
}

Integrate the Service with the Controller:

@RestController
@RequestMapping("/api/v1/pdf")
public class NewFeatureController {

    @Autowired
    private NewFeatureService newFeatureService;

    @PostMapping("/new-feature")
    public ResponseEntity<byte[]> newFeature(
        @RequestPart("fileInput") MultipartFile file,
        @RequestParam("param1") String param1) {

        byte[] result = newFeatureService.processFile(file, param1);

        return ResponseEntity.ok()
            .header("Content-Disposition", "attachment; filename=output.pdf")
            .contentType(MediaType.APPLICATION_PDF)
            .body(result);
    }
}

Multi-File Endpoints

For tools that process multiple files in one request:

@PostMapping("/merge")
public ResponseEntity<byte[]> mergePdfs(
    @RequestPart("fileInput") MultipartFile[] files) {

    // Process all files together
    byte[] merged = mergeService.mergeFiles(files);

    return ResponseEntity.ok()
        .header("Content-Disposition", "attachment; filename=merged.pdf")
        .contentType(MediaType.APPLICATION_PDF)
        .body(merged);
}

12. Best Practices

Frontend

Always use FileContext for file operations
Implement proper cleanup for PDF.js documents and blob URLs
Use the useToolOperation hook for consistent tool behavior
Follow TypeScript strict mode guidelines
Test with large files (100MB+) to ensure memory efficiency

Backend

Use PDFBox for PDF manipulation
Implement proper error handling and logging
Add Swagger documentation to all API endpoints
Use service layer for business logic
Follow Spring Boot best practices

General

Write clear commit messages
Update documentation for any API changes
Test in Docker before submitting PRs
Run ./gradlew spotlessApply to format code
Ensure all tests pass with ./test.sh

20 KiB Raw Permalink Blame History