Java Bindings
The Data Processing Library provides a Java interface to support compilers written entirely in Java. This interface exposes most of the functionality available in Scala with some exceptions.
Java specific classes are defined in the com.here.platform.data.processing.java
package and its sub-packages.
The Data Processing Library Java classes and methods have identical names to their Scala counterparts. However, method signatures may differ as Scala specific types are replaced by Java native types. The following sections describe these differences in detail.
Type Aliases
In Java, types aliases are not used. For instance, in the Scala API the InKey
and OutKey
aliases of type Partition.Key
are used when input keys and output keys are expected, respectively. However, the Java API uses a single type Key
, instead.
Type aliases in Scala:
def compileInFn(in: (InKey, InMeta)): Iterable[(OutKey, IntermediateData)] = {
Equivalent code in Java:
public Iterable<Pair<Key, IntermediateData>> compileInFn(Pair<Key, Meta> in) {
The Symbol
Class
In many Scala classes, the Symbol
class is used for methods that represent IDs or names. However, the Java counterpart uses the String
class instead.
Symbol
class in Scala:
val outKey = inKey.copy(catalog = Default.OutCatalogId, layer = Layer.Id("output-layer"))
Equivalent code in Java:
Key outKey = inKey.withCatalog(Default.OutCatalogId())
.withLayer("output-layer");
Scala Collections
Scala collections are replaced by their Java counterparts.
Collections in Scala:
def resolveFn(src: (InKey, InMeta)): Map[RefName, Set[InKey]] = {
Equivalent code in Java:
public Map<String, Set<Key>> resolveFn(Pair<Key, Meta> src) {
Tuples and Options
Tuples of 2 elements are replaced by the com.here.platform.data.processing.java.Pair
class in the Java API. The scala.Option
type is replaced by java.util.Optional
.
Option
type in Scala:
def compileOutFn(outKey: OutKey, intermediate: Iterable[IntermediateData]): Option[Payload] = {
if (intermediate.isEmpty) {
None
} else {
Some(Payload(encodeData(intermediate)))
}
}
Equivalent code in Java:
public Optional<Payload> compileOutFn(Key outKey, Iterable<IntermediateData> intermediate) {
if (intermediate.iterator().hasNext()) {
return Optional.of(new Payload(encodeData(intermediate)));
} else {
return Optional.empty();
}
}
Scala Mixins
Mixins in the com.here.platform.data.processing.compiler
package, should be combined together to implement a compiler. The Java API represents these mixins as interfaces. All traits that must be mixed-in in Scala are interfaces that you implement in Java.
Compiler mixins in Scala:
class MyCompiler
extends RefTreeCompiler[IntermediateData]
with CompileInFnWithRefs[IntermediateData]
with CompileOut1To1Fn[IntermediateData] {
Equivalent code in Java:
class MyCompiler
implements RefTreeCompiler<IntermediateData>,
CompileInFnWithRefs<IntermediateData>,
CompileOut1To1Fn<IntermediateData> {
Some Java classes provide the same functionality of multiple Scala traits mixed in together.
For instance, extending a PipelineRunner
in Scala:
object Main extends PipelineRunner with DriverSetupWithBuilder {
def configureCompiler(completeConfig: CompleteConfig,
context: DriverContext,
builder: DriverBuilder): builder.type = {
Equivalent code in Java:
class Main extends PipelineRunner {
@Override
public DriverBuilder configureCompiler(
CompleteConfig completeConfig, DriverContext context, DriverBuilder builder) {
Scala Classes without Java Specific Interface
A few Scala classes do not have Java counterparts, and can be used directly from Java. These include:
- Classes in the
com.here.platform.pipeline.logging
package, except the ContextAwareContextLogger
class, of which a Scala version (com.here.platform.pipeline.logging.scaladsl.ContextAwareLogger
) and a Java version (com.here.platform.pipeline.logging.javadsl.ContextAwareLogger
) exist. com.here.platform.data.processing.statistics.Statistic
-
com.here.platform.data.processing.driver.job.ToProcess
and subclasses
Some of these Scala classes are meant to be used in match expressions, a feature that is not available in Java.
Pattern matching in Scala:
val processingType = driverContext.jobVersions.inCatalogs(id)
val version = processingType match {
case NoChanges(v) => v
case Changes(_, v) => v
case Reprocess(v) => v
}
is replaced by a sequence of isInstanceOf
predicates in Java:
ProcessingType processingType = driverContext.jobVersions().inCatalogs().get(id);
long version;
if (processingType instanceof NoChanges) {
version = ((NoChanges) processingType).version();
} else if (processingType instanceof Changes) {
version = ((Changes) processingType).version();
} else if (processingType instanceof Reprocess) {
version = ((Reprocess) processingType).version();
} else {
throw new RuntimeException("Match error");
}
Currying
Some classes provide their functionality through curried methods, such as the LogContext
class' method withChild
. In the Java version, the body part of the method's call is an explicit lambda expression.
So the following Scala code:
val logger = new ContextAwareLogger(getClass)
LogContext.withChild("my-context-id", id) {
val payload = retriever.getPayload(in.key, in.meta)
logger.debug(s"Retrieved ${payload.length} bytes")
}
becomes:
ContextAwareLogger logger = new ContextAwareLogger(getClass());
LogContext.withChild(
"my-context-id",
id,
() -> {
Payload payload = retriever.getPayload(in.getKey(), in.getValue());
logger.debug("Retrieved {} bytes", payload.length());
});
Implicit Arguments
Implicit arguments are not supported in Java, thus they are appended to the argument list and you must provide them explicitly.
This includes Scala ClassTag
s used in some methods to carry along all type information available at compile time, to runtime.
def buildTask(driverBuilder: DriverBuilder,
compiler: Direct1ToNCompiler[IntermediateData]): DriverTask = {
val taskBuilder = driverBuilder.newTaskBuilder("task1")
taskBuilder.withDirect1ToNCompiler(compiler).build()
}
In the Java counterpart, a class object needs to be explicitly passed to withDirect1ToNCompiler
:
DriverTask buildTask(
DriverBuilder driverBuilder, Direct1ToNCompiler<IntermediateData> compiler) {
TaskBuilder taskBuilder = driverBuilder.newTaskBuilder("task1");
return taskBuilder.withDirect1ToNCompiler(compiler, IntermediateData.class).build();
}
For a complete list of features currently supported in Java, refer to the API Reference.