Skip to main content

Documentation Index

Fetch the complete documentation index at: https://mintlify.com/apache/wayang/llms.txt

Use this file to discover all available pages before exploring further.

Apache Wayang is built by the community. Whether you are fixing a typo in the docs, adding a new operator, wiring up a new execution platform, or improving the cost-based optimizer, every contribution moves the project forward. This guide walks you through the contribution workflow, development setup, build commands, and the areas where help is most welcome.
By contributing to Apache Wayang you agree to the Apache Software Foundation Code of Conduct and the Apache License, Version 2.0. All contributed code must carry the Apache license header.

​
Contribution workflow

1

Introduce yourself on the mailing list

Send a short email to the developer mailing list describing who you are and what you intend to implement or fix. This lets committers give early feedback and avoids duplicate effort.
Searching the mailing list archives first is a good idea — your topic may already be under discussion.
2

Fork the repository and create a branch

All active development happens on the main branch. Fork the repository on GitHub and create a feature branch from main:
git clone https://github.com/<your-username>/wayang.git
cd wayang
git checkout -b feature/my-contribution
3

Implement your changes

Write your code, following the coding standards enforced by Checkstyle (see below). Add tests covering your changes. If you are contributing a new operator or platform, also add or update the corresponding documentation.Make sure all builds and tests pass locally before proceeding:
./mvnw clean install
4

Open a Pull Request

Push your branch to your fork and open a Pull Request against the main branch of the upstream Apache Wayang repository. Describe what you changed and why.
5

Respond to code review

A committer will review your PR. Address any requested changes — iterate until the reviewer is satisfied. Be patient: committers are volunteers.
6

Merge

Once the reviewer approves, your PR is merged into main. Congratulations — you are now a contributor!

​
Development setup

​
Prerequisites

ToolMinimum versionNotes
Java (JDK)17The entire project compiles at --release 17; JDK 17 or later is required
Maven3.6Use the bundled ./mvnw wrapper for reproducibility
Scala2.12Managed by the build; no separate install needed
GitAny recent version
Optional, depending on the platform you want to develop against:
  • Apache Spark 3.x (for wayang-spark)
  • Apache Flink 1.x (for wayang-flink)
  • PostgreSQL or SQLite (for database platform tests)
  • Docker (for Kafka integration tests)

​
Clone the repository

git clone https://github.com/apache/wayang.git
cd wayang
The repository is a standard Maven multi-module project. The top-level pom.xml declares all modules. The bundled Maven wrapper ./mvnw (Linux/macOS) or mvnw.cmd (Windows) ensures you use the correct Maven version without a system-wide install.

​
Build commands

​
Compile a single module

When iterating on a specific module, compile only that module to keep the feedback loop short: 
./mvnw clean install -DskipTests -pl <module-name>
For example, to recompile only the Spark platform adapter: 
./mvnw clean install -DskipTests -pl wayang-platforms/wayang-spark

​
Full build (all modules, skipping tests)

./mvnw clean install -DskipTests

​
Full build with tests

Run the complete build including all unit and integration tests. Do this before opening a Pull Request — the CI gate runs this exact command. 
./mvnw clean install

​
Run tests for a specific module

./mvnw test -pl wayang-platforms/wayang-java

​
Package the standalone distribution

Produces the bin/wayang-submit launcher script and a self-contained tarball: 
./mvnw clean package -pl :wayang-assembly -Pdistribution
The output appears under wayang-assembly/target/. Extract and run: 
cd wayang-assembly/target/
tar -xvf apache-wayang-assembly-*-dist.tar.gz
cd wayang-*/
./bin/wayang-submit org.apache.wayang.applications.WordCount -- file:///path/to/input.txt
The wayang-submit script requires SPARK_HOME and HADOOP_HOME to be set in your environment. See guides/tutorial.md in the repository for full environment setup instructions.

​
Code quality

Wayang enforces code style through Checkstyle using the Google Java Style ruleset (google_checks.xml). Violations fail the build. Run Checkstyle standalone: 
./mvnw checkstyle:check
All contributed Java files must start with the Apache license header: 
/*
 * Licensed to the Apache Software Foundation (ASF) under one
 * or more contributor license agreements.  See the NOTICE file
 * distributed with this work for additional information
 * regarding copyright ownership.  The ASF licenses this file
 * to you under the Apache License, Version 2.0 (the
 * "License"); you may not use this file except in compliance
 * with the License.  You may obtain a copy of the License at
 *
 *     http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */

​
Areas for contribution

​
Documentation improvements

The fastest way to start contributing. Fix typos, improve examples, add missing explanations, or write a guide for a use case that isn’t covered yet.

Fix a doc page

Every page has an “Edit on GitHub” link. For small fixes, edit directly in the browser.

Add an example

New runnable examples — with real code and working wayang-submit commands — are always welcome.

​
New operators

Wayang operators are defined in wayang-commons/wayang-basic/src/main/java/org/apache/wayang/basic/operators/. Each operator is a logical description; platform-specific implementations live in the platform module under src/main/java/org/apache/wayang/<platform>/operators/. Steps for adding a new operator:
  1. Define the logical operator class in wayang-basic.
  2. Implement the execution logic in each target platform module.
  3. Register the mapping in the platform’s Plugin class.
  4. Add cardinality and cost estimators.
  5. Write unit tests in the platform module’s test directory.

​
New platform adapters

If you want to add support for a new execution engine, model it after an existing lightweight platform (e.g., wayang-sqlite3) and implement:
  • A Platform class
  • Operator implementations for the standard Wayang operator set
  • Mapping registrations
  • A Plugin that registers all mappings
  • Cost estimators for each operator

​
Optimizer improvements

The optimizer lives in wayang-commons/wayang-core/src/main/java/org/apache/wayang/core/optimizer/. Contributions are welcome in:
  • Better default cardinality estimators
  • Improved plan enumeration strategies
  • New cost models
  • Profiler integration for adaptive re-optimization

​
Community and getting help

Mailing lists

The developer mailing list is the primary discussion channel for design decisions, release planning, and code review.

GitHub Issues

File bug reports, feature requests, and task items. Check existing issues before opening a new one.

Community page

Find committers, mentors, and links to conference talks and publications.

Apache Code of Conduct

All interactions in the Wayang community are governed by the ASF Code of Conduct.

​
Branch strategy

BranchPurpose
mainAll new development. Fork from here.
release/*Tagged release branches. No direct commits.
Always fork from main. Release branches are managed by committers during the Apache release process and are not targets for feature PRs.

​
Becoming a committer

Contributors who demonstrate sustained, high-quality involvement in the project — through code, documentation, review, or community engagement — may be invited by the Project Management Committee (PMC) to become committers. Committers gain write access to the repository and participate in project governance. There is no formal application process; involvement and merit are observed over time. Learn more at the Apache Wayang community page.

Build docs developers (and LLMs) love

Get started for freeTalk to us