Mirrors/Stirling-PDF
1
0
Fork 0
mirror of https://github.com/Frooodle/Stirling-PDF.git synced 2026-02-17 13:52:14 +01:00
Code Issues Packages Projects Releases Wiki Activity

Toml (#5115)

Browse Source 
# Description of Changes

<!--
Please provide a summary of the changes, including:

- What was changed
- Why the change was made
- Any challenges encountered

Closes #(issue_number)
-->

---

## Checklist

### General

- [ ] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [ ] I have performed a self-review of my own code
- [ ] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### Translations (if applicable)

- [ ] I ran
[`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md)

### UI Changes (if applicable)

- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [ ] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/DeveloperGuide.md#6-testing)
for more details.
This commit is contained in:
Anthony Stirling
2025-12-03 09:57:00 +00:00
committed by GitHub
parent f72538d30f
commit 65a3eeca76
62 changed files with 3352 additions and 1947 deletions
Download Patch File Download Diff File Expand all files Collapse all files

403
.github/scripts/check_language_properties.py vendored
View File

@@ -1,403 +0,0 @@
"""
Author: Ludy87
Description: This script processes .properties files for localization checks. It compares translation files in a branch with
a reference file to ensure consistency. The script performs two main checks:
1. Verifies that the number of lines (including comments and empty lines) in the translation files matches the reference file.
2. Ensures that all keys in the translation files are present in the reference file and vice versa.
The script also provides functionality to update the translation files to match the reference file by adding missing keys and
adjusting the format.
Usage:
python check_language_properties.py --reference-file <path_to_reference_file> --branch <branch_name> [--actor <actor_name>] [--files <list_of_changed_files>]
"""
# Sample for Windows:
# python .github/scripts/check_language_properties.py --reference-file src\main\resources\messages_en_GB.properties --branch "" --files src\main\resources\messages_de_DE.properties src\main\resources\messages_uk_UA.properties
import copy
import glob
import os
import argparse
import re
def find_duplicate_keys(file_path):
"""
Identifies duplicate keys in a .properties file.
:param file_path: Path to the .properties file.
:return: List of tuples (key, first_occurrence_line, duplicate_line).
"""
keys = {}
duplicates = []
with open(file_path, "r", encoding="utf-8") as file:
for line_number, line in enumerate(file, start=1):
stripped_line = line.strip()
# Skip empty lines and comments
if not stripped_line or stripped_line.startswith("#"):
continue
# Split the line into key and value
if "=" in stripped_line:
key, _ = stripped_line.split("=", 1)
key = key.strip()
# Check if the key already exists
if key in keys:
duplicates.append((key, keys[key], line_number))
else:
keys[key] = line_number
return duplicates
# Maximum size for properties files (e.g., 200 KB)
MAX_FILE_SIZE = 200 * 1024
def parse_properties_file(file_path):
"""
Parses a .properties file and returns a structured list of its contents.
:param file_path: Path to the .properties file.
:return: List of dictionaries representing each line in the file.
"""
properties_list = []
with open(file_path, "r", encoding="utf-8") as file:
for line_number, line in enumerate(file, start=1):
stripped_line = line.strip()
# Handle empty lines
if not stripped_line:
properties_list.append(
{"line_number": line_number, "type": "empty", "content": ""}
)
continue
# Handle comments
if stripped_line.startswith("#"):
properties_list.append(
{
"line_number": line_number,
"type": "comment",
"content": stripped_line,
}
)
continue
# Handle key-value pairs
match = re.match(r"^([^=]+)=(.*)$", line)
if match:
key, value = match.groups()
properties_list.append(
{
"line_number": line_number,
"type": "entry",
"key": key.strip(),
"value": value.strip(),
}
)
return properties_list
def write_json_file(file_path, updated_properties):
"""
Writes updated properties back to the file in their original format.
:param file_path: Path to the .properties file.
:param updated_properties: List of updated properties to write.
"""
updated_lines = {entry["line_number"]: entry for entry in updated_properties}
# Sort lines by their numbers and retain comments and empty lines
all_lines = sorted(set(updated_lines.keys()))
original_format = []
for line in all_lines:
if line in updated_lines:
entry = updated_lines[line]
else:
entry = None
ref_entry = updated_lines[line]
if ref_entry["type"] in ["comment", "empty"]:
original_format.append(ref_entry)
elif entry is None:
# Add missing entries from the reference file
original_format.append(ref_entry)
elif entry["type"] == "entry":
# Replace entries with those from the current JSON
original_format.append(entry)
# Write the updated content back to the file
with open(file_path, "w", encoding="utf-8", newline="\n") as file:
for entry in original_format:
if entry["type"] == "comment":
file.write(f"{entry['content']}\n")
elif entry["type"] == "empty":
file.write(f"{entry['content']}\n")
elif entry["type"] == "entry":
file.write(f"{entry['key']}={entry['value']}\n")
def update_missing_keys(reference_file, file_list, branch=""):
"""
Updates missing keys in the translation files based on the reference file.
:param reference_file: Path to the reference .properties file.
:param file_list: List of translation files to update.
:param branch: Branch where the files are located.
"""
reference_properties = parse_properties_file(reference_file)
for file_path in file_list:
basename_current_file = os.path.basename(os.path.join(branch, file_path))
if (
basename_current_file == os.path.basename(reference_file)
or not file_path.endswith(".properties")
or not basename_current_file.startswith("messages_")
):
continue
current_properties = parse_properties_file(os.path.join(branch, file_path))
updated_properties = []
for ref_entry in reference_properties:
ref_entry_copy = copy.deepcopy(ref_entry)
for current_entry in current_properties:
if current_entry["type"] == "entry":
if ref_entry_copy["type"] != "entry":
continue
if ref_entry_copy["key"].lower() == current_entry["key"].lower():
ref_entry_copy["value"] = current_entry["value"]
updated_properties.append(ref_entry_copy)
write_json_file(os.path.join(branch, file_path), updated_properties)
def check_for_missing_keys(reference_file, file_list, branch):
update_missing_keys(reference_file, file_list, branch)
def read_properties(file_path):
if os.path.isfile(file_path) and os.path.exists(file_path):
with open(file_path, "r", encoding="utf-8") as file:
return file.read().splitlines()
return [""]
def check_for_differences(reference_file, file_list, branch, actor):
reference_branch = reference_file.split("/")[0]
basename_reference_file = os.path.basename(reference_file)
report = []
report.append(f"#### 🔄 Reference Branch: `{reference_branch}`")
reference_lines = read_properties(reference_file)
has_differences = False
only_reference_file = True
file_arr = file_list
if len(file_list) == 1:
file_arr = file_list[0].split()
base_dir = os.path.abspath(
os.path.join(os.getcwd(), "app", "core", "src", "main", "resources")
)
for file_path in file_arr:
file_normpath = os.path.normpath(file_path)
absolute_path = os.path.abspath(file_normpath)
# Verify that file is within the expected directory
if not absolute_path.startswith(base_dir):
raise ValueError(f"Unsafe file found: {file_normpath}")
# Verify file size before processing
if os.path.getsize(os.path.join(branch, file_normpath)) > MAX_FILE_SIZE:
raise ValueError(
f"The file {file_normpath} is too large and could pose a security risk."
)
basename_current_file = os.path.basename(os.path.join(branch, file_normpath))
if (
basename_current_file == basename_reference_file
or (
# only local windows command
not file_normpath.startswith(
os.path.join(
"", "app", "core", "src", "main", "resources", "messages_"
)
)
and not file_normpath.startswith(
os.path.join(
os.getcwd(),
"app",
"core",
"src",
"main",
"resources",
"messages_",
)
)
)
or not file_normpath.endswith(".properties")
or not basename_current_file.startswith("messages_")
):
continue
only_reference_file = False
report.append(f"#### 📃 **File Check:** `{basename_current_file}`")
current_lines = read_properties(os.path.join(branch, file_path))
reference_line_count = len(reference_lines)
current_line_count = len(current_lines)
if reference_line_count != current_line_count:
report.append("")
report.append("1. **Test Status:** ❌ **_Failed_**")
report.append(" - **Issue:**")
has_differences = True
if reference_line_count > current_line_count:
report.append(
f" - **_Mismatched line count_**: {reference_line_count} (reference) vs {current_line_count} (current). Comments, empty lines, or translation strings are missing."
)
elif reference_line_count < current_line_count:
report.append(
f" - **_Too many lines_**: {reference_line_count} (reference) vs {current_line_count} (current). Please verify if there is an additional line that needs to be removed."
)
else:
report.append("1. **Test Status:** ✅ **_Passed_**")
# Check for missing or extra keys
current_keys = []
reference_keys = []
for line in current_lines:
if not line.startswith("#") and line != "" and "=" in line:
key, _ = line.split("=", 1)
current_keys.append(key)
for line in reference_lines:
if not line.startswith("#") and line != "" and "=" in line:
key, _ = line.split("=", 1)
reference_keys.append(key)
current_keys_set = set(current_keys)
reference_keys_set = set(reference_keys)
missing_keys = current_keys_set.difference(reference_keys_set)
extra_keys = reference_keys_set.difference(current_keys_set)
missing_keys_list = list(missing_keys)
extra_keys_list = list(extra_keys)
if missing_keys_list or extra_keys_list:
has_differences = True
missing_keys_str = "`, `".join(missing_keys_list)
extra_keys_str = "`, `".join(extra_keys_list)
report.append("2. **Test Status:** ❌ **_Failed_**")
report.append(" - **Issue:**")
if missing_keys_list:
spaces_keys_list = []
for key in missing_keys_list:
if " " in key:
spaces_keys_list.append(key)
if spaces_keys_list:
spaces_keys_str = "`, `".join(spaces_keys_list)
report.append(
f" - **_Keys containing unnecessary spaces_**: `{spaces_keys_str}`!"
)
report.append(
f" - **_Extra keys in `{basename_current_file}`_**: `{missing_keys_str}` that are not present in **_`{basename_reference_file}`_**."
)
if extra_keys_list:
report.append(
f" - **_Missing keys in `{basename_reference_file}`_**: `{extra_keys_str}` that are not present in **_`{basename_current_file}`_**."
)
else:
report.append("2. **Test Status:** ✅ **_Passed_**")
if find_duplicate_keys(os.path.join(branch, file_normpath)):
has_differences = True
output = "\n".join(
[
f" - `{key}`: first at line {first}, duplicate at `line {duplicate}`"
for key, first, duplicate in find_duplicate_keys(
os.path.join(branch, file_normpath)
)
]
)
report.append("3. **Test Status:** ❌ **_Failed_**")
report.append(" - **Issue:**")
report.append(" - duplicate entries were found:")
report.append(output)
else:
report.append("3. **Test Status:** ✅ **_Passed_**")
report.append("")
report.append("---")
report.append("")
if has_differences:
report.append("## ❌ Overall Check Status: **_Failed_**")
report.append("")
report.append(
f"@{actor} please check your translation if it conforms to the standard. Follow the format of [messages_en_GB.properties](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/app/core/src/main/resources/messages_en_GB.properties)"
)
else:
report.append("## ✅ Overall Check Status: **_Success_**")
report.append("")
report.append(
f"Thanks @{actor} for your help in keeping the translations up to date."
)
if not only_reference_file:
print("\n".join(report))
if __name__ == "__main__":
parser = argparse.ArgumentParser(description="Find missing keys")
parser.add_argument(
"--actor",
required=False,
help="Actor from PR.",
)
parser.add_argument(
"--reference-file",
required=True,
help="Path to the reference file.",
)
parser.add_argument(
"--branch",
type=str,
required=True,
help="Branch name.",
)
parser.add_argument(
"--check-file",
type=str,
required=False,
help="List of changed files, separated by spaces.",
)
parser.add_argument(
"--files",
nargs="+",
required=False,
help="List of changed files, separated by spaces.",
)
args = parser.parse_args()
# Sanitize --actor input to avoid injection attacks
if args.actor:
args.actor = re.sub(r"[^a-zA-Z0-9_\\-]", "", args.actor)
# Sanitize --branch input to avoid injection attacks
if args.branch:
args.branch = re.sub(r"[^a-zA-Z0-9\\-]", "", args.branch)
file_list = args.files
if file_list is None:
if args.check_file:
file_list = [args.check_file]
else:
file_list = glob.glob(
os.path.join(
os.getcwd(),
"app",
"core",
"src",
"main",
"resources",
"messages_*.properties",
)
)
update_missing_keys(args.reference_file, file_list)
else:
check_for_differences(args.reference_file, file_list, args.branch, args.actor)

68
.github/scripts/check_language_json.py → .github/scripts/check_language_toml.py vendored
View File

@@ -1,6 +1,6 @@
"""
Author: Ludy87
Description: This script processes JSON translation files for localization checks. It compares translation files in a branch with
Description: This script processes TOML translation files for localization checks. It compares translation files in a branch with
a reference file to ensure consistency. The script performs two main checks:
1. Verifies that the number of translation keys in the translation files matches the reference file.
2. Ensures that all keys in the translation files are present in the reference file and vice versa.
@@ -9,10 +9,10 @@ The script also provides functionality to update the translation files to match
adjusting the format.
Usage:
python check_language_json.py --reference-file <path_to_reference_file> --branch <branch_name> [--actor <actor_name>] [--files <list_of_changed_files>]
python check_language_toml.py --reference-file <path_to_reference_file> --branch <branch_name> [--actor <actor_name>] [--files <list_of_changed_files>]
"""
# Sample for Windows:
# python .github/scripts/check_language_json.py --reference-file frontend/public/locales/en-GB/translation.json --branch "" --files frontend/public/locales/de-DE/translation.json frontend/public/locales/fr-FR/translation.json
# python .github/scripts/check_language_toml.py --reference-file frontend/public/locales/en-GB/translation.toml --branch "" --files frontend/public/locales/de-DE/translation.toml frontend/public/locales/fr-FR/translation.toml
import copy
import glob
@@ -20,12 +20,14 @@ import os
import argparse
import re
import json
import tomllib # Python 3.11+ (stdlib)
import tomli_w # For writing TOML files
def find_duplicate_keys(file_path, keys=None, prefix=""):
"""
Identifies duplicate keys in a JSON file (including nested keys).
:param file_path: Path to the JSON file.
Identifies duplicate keys in a TOML file (including nested keys).
:param file_path: Path to the TOML file.
:param keys: Dictionary to track keys (used for recursion).
:param prefix: Prefix for nested keys.
:return: List of tuples (key, first_occurrence_path, duplicate_path).
@@ -35,8 +37,9 @@ def find_duplicate_keys(file_path, keys=None, prefix=""):
duplicates = []
with open(file_path, "r", encoding="utf-8") as file:
data = json.load(file)
# Load TOML file
with open(file_path, 'rb') as file:
data = tomllib.load(file)
def process_dict(obj, current_prefix=""):
for key, value in obj.items():
@@ -54,18 +57,18 @@ def find_duplicate_keys(file_path, keys=None, prefix=""):
return duplicates
# Maximum size for JSON files (e.g., 500 KB)
# Maximum size for TOML files (e.g., 500 KB)
MAX_FILE_SIZE = 500 * 1024
def parse_json_file(file_path):
def parse_toml_file(file_path):
"""
Parses a JSON translation file and returns a flat dictionary of all keys.
:param file_path: Path to the JSON file.
Parses a TOML translation file and returns a flat dictionary of all keys.
:param file_path: Path to the TOML file.
:return: Dictionary with flattened keys.
"""
with open(file_path, "r", encoding="utf-8") as file:
data = json.load(file)
with open(file_path, 'rb') as file:
data = tomllib.load(file)
def flatten_dict(d, parent_key="", sep="."):
items = {}
@@ -99,38 +102,37 @@ def unflatten_dict(d, sep="."):
return result
def write_json_file(file_path, updated_properties):
def write_toml_file(file_path, updated_properties):
"""
Writes updated properties back to the JSON file.
:param file_path: Path to the JSON file.
Writes updated properties back to the TOML file.
:param file_path: Path to the TOML file.
:param updated_properties: Dictionary of updated properties to write.
"""
nested_data = unflatten_dict(updated_properties)
with open(file_path, "w", encoding="utf-8", newline="\n") as file:
json.dump(nested_data, file, ensure_ascii=False, indent=2)
file.write("\n") # Add trailing newline
with open(file_path, "wb") as file:
tomli_w.dump(nested_data, file)
def update_missing_keys(reference_file, file_list, branch=""):
"""
Updates missing keys in the translation files based on the reference file.
:param reference_file: Path to the reference JSON file.
:param reference_file: Path to the reference TOML file.
:param file_list: List of translation files to update.
:param branch: Branch where the files are located.
"""
reference_properties = parse_json_file(reference_file)
reference_properties = parse_toml_file(reference_file)
for file_path in file_list:
basename_current_file = os.path.basename(os.path.join(branch, file_path))
if (
basename_current_file == os.path.basename(reference_file)
or not file_path.endswith(".json")
or not file_path.endswith(".toml")
or not os.path.dirname(file_path).endswith("locales")
):
continue
current_properties = parse_json_file(os.path.join(branch, file_path))
current_properties = parse_toml_file(os.path.join(branch, file_path))
updated_properties = {}
for ref_key, ref_value in reference_properties.items():
@@ -141,16 +143,16 @@ def update_missing_keys(reference_file, file_list, branch=""):
# Add missing key with reference value
updated_properties[ref_key] = ref_value
write_json_file(os.path.join(branch, file_path), updated_properties)
write_toml_file(os.path.join(branch, file_path), updated_properties)
def check_for_missing_keys(reference_file, file_list, branch):
update_missing_keys(reference_file, file_list, branch)
def read_json_keys(file_path):
def read_toml_keys(file_path):
if os.path.isfile(file_path) and os.path.exists(file_path):
return parse_json_file(file_path)
return parse_toml_file(file_path)
return {}
@@ -160,7 +162,7 @@ def check_for_differences(reference_file, file_list, branch, actor):
report = []
report.append(f"#### 🔄 Reference Branch: `{reference_branch}`")
reference_keys = read_json_keys(reference_file)
reference_keys = read_toml_keys(reference_file)
has_differences = False
only_reference_file = True
@@ -197,12 +199,12 @@ def check_for_differences(reference_file, file_list, branch, actor):
):
continue
if not file_normpath.endswith(".json") or basename_current_file != "translation.json":
if not file_normpath.endswith(".toml") or basename_current_file != "translation.toml":
continue
only_reference_file = False
report.append(f"#### 📃 **File Check:** `{locale_dir}/{basename_current_file}`")
current_keys = read_json_keys(os.path.join(branch, file_path))
current_keys = read_toml_keys(os.path.join(branch, file_path))
reference_key_count = len(reference_keys)
current_key_count = len(current_keys)
@@ -272,7 +274,7 @@ def check_for_differences(reference_file, file_list, branch, actor):
report.append("## ❌ Overall Check Status: **_Failed_**")
report.append("")
report.append(
f"@{actor} please check your translation if it conforms to the standard. Follow the format of [en-GB/translation.json](https://github.com/Stirling-Tools/Stirling-PDF/blob/V2/frontend/public/locales/en-GB/translation.json)"
f"@{actor} please check your translation if it conforms to the standard. Follow the format of [en-GB/translation.toml](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/frontend/public/locales/en-GB/translation.toml)"
)
else:
report.append("## ✅ Overall Check Status: **_Success_**")
@@ -286,7 +288,7 @@ def check_for_differences(reference_file, file_list, branch, actor):
if __name__ == "__main__":
parser = argparse.ArgumentParser(description="Find missing keys")
parser = argparse.ArgumentParser(description="Find missing keys in TOML translation files")
parser.add_argument(
"--actor",
required=False,
@@ -337,9 +339,9 @@ if __name__ == "__main__":
"public",
"locales",
"*",
"translation.json",
"translation.toml",
)
)
update_missing_keys(args.reference_file, file_list)
else:
check_for_differences(args.reference_file, file_list, args.branch, args.actor)
check_for_differences(args.reference_file, file_list, args.branch, args.actor)

82
.github/workflows/check_properties.yml → .github/workflows/check_toml.yml vendored
View File

@@ -1,19 +1,14 @@
name: Check Properties Files on PR
name: Check TOML Translation Files on PR
# This workflow validates TOML translation files
on:
pull_request_target:
types: [opened, synchronize, reopened]
paths:
- "app/core/src/main/resources/messages_*.properties"
- "frontend/public/locales/*/translation.toml"
# cancel in-progress jobs if a new job is triggered
# This is useful to avoid running multiple builds for the same branch if a new commit is pushed
# or a pull request is updated.
# It helps to save resources and time by ensuring that only the latest commit is built and tested
# This is particularly useful for long-running jobs that may take a while to complete.
# The `group` is set to a combination of the workflow name, event name, and branch name.
# This ensures that jobs are grouped by the workflow and branch, allowing for cancellation of
# in-progress jobs when a new commit is pushed to the same branch or a new pull request is opened.
concurrency:
group: ${{ github.workflow }}-${{ github.event_name }}-${{ github.event.pull_request.number || github.ref_name || github.ref }}
cancel-in-progress: true
@@ -73,22 +68,22 @@ jobs:
run: |
echo "Fetching PR changed files..."
echo "Getting list of changed files from PR..."
# Check if PR number exists
if [ -z "${{ steps.get-pr-data.outputs.pr_number }}" ]; then
echo "Error: PR number is empty"
exit 1
fi
# Get changed files and filter for properties files, handle case where no matches are found
gh pr view ${{ steps.get-pr-data.outputs.pr_number }} --json files -q ".files[].path" | grep -E '^app/core/src/main/resources/messages_[a-zA-Z_]{2}_[a-zA-Z_]{2,7}\.properties$' > changed_files.txt || echo "No matching properties files found in PR"
# Check if any files were found
if [ ! -s changed_files.txt ]; then
echo "No properties files changed in this PR"
echo "Workflow will exit early as no relevant files to check"
exit 0
fi
echo "Found $(wc -l < changed_files.txt) matching properties files"
# Check if PR number exists
if [ -z "${{ steps.get-pr-data.outputs.pr_number }}" ]; then
echo "Error: PR number is empty"
exit 1
fi
# Get changed files and filter for TOML translation files
gh pr view ${{ steps.get-pr-data.outputs.pr_number }} --json files -q ".files[].path" | grep -E '^frontend/public/locales/[a-zA-Z-]+/translation\.toml$' > changed_files.txt || echo "No matching TOML files found in PR"
# Check if any files were found
if [ ! -s changed_files.txt ]; then
echo "No TOML translation files changed in this PR"
echo "Workflow will exit early as no relevant files to check"
exit 0
fi
echo "Found $(wc -l < changed_files.txt) matching TOML files"
- name: Determine reference file test
- name: Determine reference file
id: determine-file
uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8.0.0
with:
@@ -125,11 +120,11 @@ jobs:
pull_number: prNumber,
});
// Filter for relevant files based on the PR changes
// Filter for relevant TOML files based on the PR changes
const changedFiles = files
.filter(file =>
file.status !== "removed" &&
/^app\/core\/src\/main\/resources\/messages_[a-zA-Z_]{2}_[a-zA-Z_]{2,7}\.properties$/.test(file.filename)
/^frontend\/public\/locales\/[a-zA-Z-]+\/translation\.toml$/.test(file.filename)
)
.map(file => file.filename);
@@ -169,16 +164,16 @@ jobs:
// Determine reference file
let referenceFilePath;
if (changedFiles.includes("app/core/src/main/resources/messages_en_GB.properties")) {
if (changedFiles.includes("frontend/public/locales/en-GB/translation.toml")) {
console.log("Using PR branch reference file.");
const { data: fileContent } = await github.rest.repos.getContent({
owner: prRepoOwner,
repo: prRepoName,
path: "app/core/src/main/resources/messages_en_GB.properties",
path: "frontend/public/locales/en-GB/translation.toml",
ref: branch,
});
referenceFilePath = "pr-branch-messages_en_GB.properties";
referenceFilePath = "pr-branch-translation-en-GB.toml";
const content = Buffer.from(fileContent.content, "base64").toString("utf-8");
fs.writeFileSync(referenceFilePath, content);
} else {
@@ -186,11 +181,11 @@ jobs:
const { data: fileContent } = await github.rest.repos.getContent({
owner: repoOwner,
repo: repoName,
path: "app/core/src/main/resources/messages_en_GB.properties",
path: "frontend/public/locales/en-GB/translation.toml",
ref: "main",
});
referenceFilePath = "main-branch-messages_en_GB.properties";
referenceFilePath = "main-branch-translation-en-GB.toml";
const content = Buffer.from(fileContent.content, "base64").toString("utf-8");
fs.writeFileSync(referenceFilePath, content);
}
@@ -198,11 +193,20 @@ jobs:
console.log(`Reference file path: ${referenceFilePath}`);
core.exportVariable("REFERENCE_FILE", referenceFilePath);
- name: Set up Python
uses: actions/setup-python@e797f83bcb11b83ae66e0230d6156d7c80228e7c # v6.0.0
with:
python-version: "3.12"
- name: Install Python dependencies
run: |
pip install tomli-w
- name: Run Python script to check files
id: run-check
run: |
echo "Running Python script to check files..."
python .github/scripts/check_language_properties.py \
echo "Running Python script to check TOML files..."
python .github/scripts/check_language_toml.py \
--actor ${{ github.event.pull_request.user.login }} \
--reference-file "${REFERENCE_FILE}" \
--branch "pr-branch" \
@@ -213,7 +217,7 @@ jobs:
id: capture-output
run: |
if [ -f result.txt ] && [ -s result.txt ]; then
echo "Test, capturing output..."
echo "Capturing output..."
SCRIPT_OUTPUT=$(cat result.txt)
echo "SCRIPT_OUTPUT<<EOF" >> $GITHUB_ENV
echo "$SCRIPT_OUTPUT" >> $GITHUB_ENV
@@ -227,7 +231,7 @@ jobs:
echo "FAIL_JOB=false" >> $GITHUB_ENV
fi
else
echo "No update found."
echo "No output found."
echo "SCRIPT_OUTPUT=" >> $GITHUB_ENV
echo "FAIL_JOB=false" >> $GITHUB_ENV
fi
@@ -249,7 +253,7 @@ jobs:
issue_number: issueNumber
});
const comment = comments.data.find(c => c.body.includes("## 🚀 Translation Verification Summary"));
const comment = comments.data.find(c => c.body.includes("## 🌐 TOML Translation Verification Summary"));
// Only update or create comments by the action user
const expectedActor = "${{ steps.setup-bot.outputs.app-slug }}[bot]";
@@ -260,7 +264,7 @@ jobs:
owner: repoOwner,
repo: repoName,
comment_id: comment.id,
body: `## 🚀 Translation Verification Summary\n\n\n${SCRIPT_OUTPUT}\n`
body: `## 🌐 TOML Translation Verification Summary\n\n\n${SCRIPT_OUTPUT}\n`
});
console.log("Updated existing comment.");
} else if (!comment) {
@@ -269,7 +273,7 @@ jobs:
owner: repoOwner,
repo: repoName,
issue_number: issueNumber,
body: `## 🚀 Translation Verification Summary\n\n\n${SCRIPT_OUTPUT}\n`
body: `## 🌐 TOML Translation Verification Summary\n\n\n${SCRIPT_OUTPUT}\n`
});
console.log("Created new comment.");
} else {
@@ -287,6 +291,6 @@ jobs:
run: |
echo "Cleaning up temporary files..."
rm -rf pr-branch
rm -f pr-branch-messages_en_GB.properties main-branch-messages_en_GB.properties changed_files.txt result.txt
rm -f pr-branch-translation-en-GB.toml main-branch-translation-en-GB.toml changed_files.txt result.txt
echo "Cleanup complete."
continue-on-error: true # Ensure cleanup runs even if previous steps fail

122
.github/workflows/sync_files.yml vendored
View File

@@ -1,122 +0,0 @@
name: Sync Files
on:
workflow_dispatch:
push:
branches:
- main
paths:
- "build.gradle"
- "README.md"
- "app/core/src/main/resources/messages_*.properties"
- "app/core/src/main/resources/static/3rdPartyLicenses.json"
- "scripts/ignore_translation.toml"
# cancel in-progress jobs if a new job is triggered
# This is useful to avoid running multiple builds for the same branch if a new commit is pushed
# or a pull request is updated.
# It helps to save resources and time by ensuring that only the latest commit is built and tested
# This is particularly useful for long-running jobs that may take a while to complete.
# The `group` is set to a combination of the workflow name, event name, and branch name.
# This ensures that jobs are grouped by the workflow and branch, allowing for cancellation of
# in-progress jobs when a new commit is pushed to the same branch or a new pull request is opened.
concurrency:
group: ${{ github.workflow }}-${{ github.event_name }}-${{ github.ref_name || github.ref }}
cancel-in-progress: true
permissions:
contents: read
jobs:
sync-files:
runs-on: ubuntu-latest
env:
# Prevents sdist builds → no tar extraction
PIP_ONLY_BINARY: ":all:"
PIP_DISABLE_PIP_VERSION_CHECK: "1"
steps:
- name: Harden Runner
uses: step-security/harden-runner@f4a75cfd619ee5ce8d5b864b0d183aff3c69b55a # v2.13.1
with:
egress-policy: audit
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
- name: Setup GitHub App Bot
id: setup-bot
uses: ./.github/actions/setup-bot
with:
app-id: ${{ secrets.GH_APP_ID }}
private-key: ${{ secrets.GH_APP_PRIVATE_KEY }}
- name: Set up Python
uses: actions/setup-python@e797f83bcb11b83ae66e0230d6156d7c80228e7c # v6.0.0
with:
python-version: "3.12"
cache: "pip" # caching pip dependencies
- name: Sync translation property files
run: |
python .github/scripts/check_language_properties.py --reference-file "app/core/src/main/resources/messages_en_GB.properties" --branch main
- name: Commit translation files
run: |
git add app/core/src/main/resources/messages_*.properties
git diff --staged --quiet || git commit -m ":memo: Sync translation files" || echo "No changes detected"
- name: Install dependencies
# Wheels-only + Hash-Pinning
run: |
pip install --require-hashes --only-binary=:all: -r ./.github/scripts/requirements_sync_readme.txt
- name: Sync README.md
run: |
python scripts/counter_translation.py
- name: Run git add
run: |
git add README.md scripts/ignore_translation.toml
git diff --staged --quiet || git commit -m ":memo: Sync README.md & scripts/ignore_translation.toml" || echo "No changes detected"
- name: Create Pull Request
if: always()
uses: peter-evans/create-pull-request@271a8d0340265f705b14b6d32b9829c1cb33d45e # v7.0.8
with:
token: ${{ steps.setup-bot.outputs.token }}
commit-message: Update files
committer: ${{ steps.setup-bot.outputs.committer }}
author: ${{ steps.setup-bot.outputs.committer }}
signoff: true
branch: sync_readme
title: ":globe_with_meridians: Sync Translations + Update README Progress Table"
body: |
### Description of Changes
This Pull Request was automatically generated to synchronize updates to translation files and documentation. Below are the details of the changes made:
#### **1. Synchronization of Translation Files**
- Updated translation files (`messages_*.properties`) to reflect changes in the reference file `messages_en_GB.properties`.
- Ensured consistency and synchronization across all supported language files.
- Highlighted any missing or incomplete translations.
#### **2. Update README.md**
- Generated the translation progress table in `README.md`.
- Added a summary of the current translation status for all supported languages.
- Included up-to-date statistics on translation coverage.
#### **Why these changes are necessary**
- Keeps translation files aligned with the latest reference updates.
- Ensures the documentation reflects the current translation progress.
---
Auto-generated by [create-pull-request][1].
[1]: https://github.com/peter-evans/create-pull-request
draft: false
delete-branch: true
labels: github-actions
sign-commits: true
add-paths: |
README.md
app/core/src/main/resources/messages_*.properties

38
.github/workflows/sync_files_v2.yml vendored
View File

@@ -1,15 +1,15 @@
name: Sync Files V2
name: Sync Files (TOML)
on:
workflow_dispatch:
push:
branches:
- V2
- main
- syncLangTest
paths:
- "build.gradle"
- "README.md"
- "frontend/public/locales/*/translation.json"
- "frontend/public/locales/*/translation.toml"
- "app/core/src/main/resources/static/3rdPartyLicenses.json"
- "scripts/ignore_translation.toml"
@@ -52,21 +52,25 @@ jobs:
python-version: "3.12"
cache: "pip" # caching pip dependencies
- name: Sync translation JSON files
- name: Install Python dependencies
run: |
python .github/scripts/check_language_json.py --reference-file "frontend/public/locales/en-GB/translation.json" --branch V2
pip install tomli-w
- name: Sync translation TOML files
run: |
python .github/scripts/check_language_toml.py --reference-file "frontend/public/locales/en-GB/translation.toml" --branch main
- name: Commit translation files
run: |
git add frontend/public/locales/*/translation.json
git diff --staged --quiet || git commit -m ":memo: Sync translation files" || echo "No changes detected"
git add frontend/public/locales/*/translation.toml
git diff --staged --quiet || git commit -m ":memo: Sync translation files (TOML)" || echo "No changes detected"
- name: Install dependencies
- name: Install README dependencies
run: pip install --require-hashes -r ./.github/scripts/requirements_sync_readme.txt
- name: Sync README.md
run: |
python scripts/counter_translation_v2.py
python scripts/counter_translation_v3.py
- name: Run git add
run: |
@@ -82,21 +86,22 @@ jobs:
committer: ${{ steps.setup-bot.outputs.committer }}
author: ${{ steps.setup-bot.outputs.committer }}
signoff: true
branch: sync_readme_v2
base: V2
title: ":globe_with_meridians: [V2] Sync Translations + Update README Progress Table"
branch: sync_readme_v3
base: main
title: ":globe_with_meridians: Sync Translations + Update README Progress Table"
body: |
### Description of Changes
This Pull Request was automatically generated to synchronize updates to translation files and documentation for the **V2 branch**. Below are the details of the changes made:
This Pull Request was automatically generated to synchronize updates to translation files and documentation. Below are the details of the changes made:
#### **1. Synchronization of Translation Files**
- Updated translation files (`frontend/public/locales/*/translation.json`) to reflect changes in the reference file `en-GB/translation.json`.
- Updated translation files (`frontend/public/locales/*/translation.toml`) to reflect changes in the reference file `en-GB/translation.toml`.
- Ensured consistency and synchronization across all supported language files.
- Highlighted any missing or incomplete translations.
- **Format**: TOML
#### **2. Update README.md**
- Generated the translation progress table in `README.md`.
- Generated the translation progress table in `README.md` using `counter_translation_v3.py`.
- Added a summary of the current translation status for all supported languages.
- Included up-to-date statistics on translation coverage.
@@ -115,4 +120,5 @@ jobs:
sign-commits: true
add-paths: |
README.md
frontend/public/locales/*/translation.json
frontend/public/locales/*/translation.toml
scripts/ignore_translation.toml