Pipeline Environments supported by SDK 2.54.3

The platform offers different pipeline runtime environments. The SDK 2.54.3 is compatible with two of these environments. When you create a pipeline to deploy your SDK-based application in the platform, you should select one of these two depending on the processing framework you chose during development of your application:

The following lists the dependencies provided by these two pipeline environments. Note, that these packages are loaded by the pipeline's Java Virtual Machine (JVM) and take precedence over any other application-provided package.

For more details see how to Include the SDK in Your Project.

Environment for Batch Processing 3.0

Group ID Artifact ID Version
aopalliance aopalliance 1.0
ch.qos.logback logback-classic 1.2.3
ch.qos.logback logback-core 1.2.3
com.amazonaws aws-java-sdk 1.7.4
com.carrotsearch hppc 0.7.2
com.chuusai shapeless_2.12 2.3.2
com.clearspring.analytics stream 2.7.0
com.esotericsoftware kryo-shaded 4.0.2
com.esotericsoftware minlog 1.3.0
com.esotericsoftware reflectasm 1.11.3
com.fasterxml.jackson.core jackson-annotations 2.6.7
com.fasterxml.jackson.core jackson-core 2.6.7
com.fasterxml.jackson.core jackson-databind 2.6.7.3
com.fasterxml.jackson.dataformat jackson-dataformat-yaml 2.6.7
com.fasterxml.jackson.module jackson-module-jaxb-annotations 2.6.7
com.fasterxml.jackson.module jackson-module-paranamer 2.7.9
com.fasterxml.jackson.module jackson-module-scala_2.12 2.6.7.1
com.github.fommil.netlib core 1.1.2
com.github.luben zstd-jni 1.3.2-2
com.github.mifmif generex 1.0.2
com.github.rwl jtransforms 2.4.0
com.google.code.findbugs jsr305 1.3.9
com.google.code.gson gson 2.2.4
com.google.guava guava 14.0.1
com.google.inject guice 3.0
com.google.inject.extensions guice-servlet 3.0
com.google.protobuf protobuf-java 2.5.0
com.ning compress-lzf 1.0.3
com.squareup.okhttp3 logging-interceptor 3.12.0
com.squareup.okhttp3 okhttp 3.12.10
com.squareup.okio okio 1.15.0
com.thoughtworks.paranamer paranamer 2.8
com.twitter chill-java 0.9.3
com.twitter chill_2.12 0.9.3
com.univocity univocity-parsers 2.7.3
com.vlkan flatbuffers 1.2.0-3f79e055
commons-beanutils commons-beanutils 1.7.0
commons-beanutils commons-beanutils-core 1.8.0
commons-cli commons-cli 1.2
commons-codec commons-codec 1.10
commons-collections commons-collections 3.2.2
commons-configuration commons-configuration 1.6
commons-digester commons-digester 1.8
commons-httpclient commons-httpclient 3.1
commons-io commons-io 2.4
commons-lang commons-lang 2.6
commons-logging commons-logging 1.1.3
commons-net commons-net 3.1
dk.brics.automaton automaton 1.11-8
io.airlift aircompressor 0.10
io.fabric8 kubernetes-client 4.6.1
io.fabric8 kubernetes-model 4.6.1
io.fabric8 kubernetes-model-common 4.6.1
io.fabric8 zjsonpatch 0.3.0
io.netty netty 3.9.9.Final
io.netty netty-all 4.1.47.Final
javax.activation activation 1.1.1
javax.annotation javax.annotation-api 1.2
javax.inject javax.inject 1
javax.servlet javax.servlet-api 3.1.0
javax.servlet.jsp jsp-api 2.1
javax.validation validation-api 1.1.0.Final
javax.ws.rs javax.ws.rs-api 2.0.1
javax.xml.bind jaxb-api 2.2.2
javax.xml.stream stax-api 1.0-2
joda-time joda-time 2.9.3
log4j log4j 1.2.17
net.razorvine pyrolite 4.13
net.sf.opencsv opencsv 2.3
net.sf.py4j py4j 0.10.7
net.sourceforge.f2j arpack_combined_all 0.1
org.antlr antlr4-runtime 4.7
org.apache.arrow arrow-format 0.10.0
org.apache.arrow arrow-memory 0.10.0
org.apache.arrow arrow-vector 0.10.0
org.apache.avro avro 1.8.2
org.apache.avro avro-guava-dependencies 1.8.2
org.apache.avro avro-ipc 1.8.2
org.apache.avro avro-mapred 1.8.2
org.apache.commons commons-compress 1.8.1
org.apache.commons commons-crypto 1.0.0
org.apache.commons commons-lang3 3.5
org.apache.commons commons-math3 3.4.1
org.apache.curator curator-client 2.7.1
org.apache.curator curator-framework 2.7.1
org.apache.curator curator-recipes 2.7.1
org.apache.directory.api api-asn1-api 1.0.0-M20
org.apache.directory.api api-util 1.0.0-M20
org.apache.directory.server apacheds-i18n 2.0.0-M15
org.apache.directory.server apacheds-kerberos-codec 2.0.0-M15
org.apache.hadoop hadoop-annotations 2.7.7
org.apache.hadoop hadoop-auth 2.7.7
org.apache.hadoop hadoop-aws 2.7.7
org.apache.hadoop hadoop-client 2.7.7
org.apache.hadoop hadoop-common 2.7.7
org.apache.hadoop hadoop-hdfs 2.7.7
org.apache.hadoop hadoop-mapreduce-client-app 2.7.7
org.apache.hadoop hadoop-mapreduce-client-common 2.7.7
org.apache.hadoop hadoop-mapreduce-client-core 2.7.7
org.apache.hadoop hadoop-mapreduce-client-jobclient 2.7.7
org.apache.hadoop hadoop-mapreduce-client-shuffle 2.7.7
org.apache.hadoop hadoop-yarn-api 2.7.7
org.apache.hadoop hadoop-yarn-client 2.7.7
org.apache.hadoop hadoop-yarn-common 2.7.7
org.apache.hadoop hadoop-yarn-server-common 2.7.7
org.apache.htrace htrace-core 3.1.0-incubating
org.apache.httpcomponents httpclient 4.5.6
org.apache.httpcomponents httpcore 4.4.10
org.apache.ivy ivy 2.4.0
org.apache.mesos mesos 1.4.0
org.apache.orc orc-core 1.5.5
org.apache.orc orc-mapreduce 1.5.5
org.apache.orc orc-shims 1.5.5
org.apache.parquet parquet-column 1.10.1
org.apache.parquet parquet-common 1.10.1
org.apache.parquet parquet-encoding 1.10.1
org.apache.parquet parquet-format 2.4.0
org.apache.parquet parquet-hadoop 1.10.1
org.apache.parquet parquet-jackson 1.10.1
org.apache.spark spark-catalyst_2.12 2.4.7
org.apache.spark spark-core_2.12 2.4.7
org.apache.spark spark-graphx_2.12 2.4.7
org.apache.spark spark-kubernetes_2.12 2.4.7
org.apache.spark spark-kvstore_2.12 2.4.7
org.apache.spark spark-launcher_2.12 2.4.7
org.apache.spark spark-mesos_2.12 2.4.7
org.apache.spark spark-mllib-local_2.12 2.4.7
org.apache.spark spark-mllib_2.12 2.4.7
org.apache.spark spark-network-common_2.12 2.4.7
org.apache.spark spark-network-shuffle_2.12 2.4.7
org.apache.spark spark-repl_2.12 2.4.7
org.apache.spark spark-sketch_2.12 2.4.7
org.apache.spark spark-sql_2.12 2.4.7
org.apache.spark spark-streaming_2.12 2.4.7
org.apache.spark spark-tags_2.12 2.4.7
org.apache.spark spark-unsafe_2.12 2.4.7
org.apache.spark spark-yarn_2.12 2.4.7
org.apache.xbean xbean-asm6-shaded 4.8
org.apache.zookeeper zookeeper 3.4.6
org.codehaus.jackson jackson-core-asl 1.9.13
org.codehaus.jackson jackson-jaxrs 1.9.13
org.codehaus.jackson jackson-mapper-asl 1.9.13
org.codehaus.jackson jackson-xc 1.9.13
org.codehaus.janino commons-compiler 3.0.16
org.codehaus.janino janino 3.0.16
org.fusesource.leveldbjni leveldbjni-all 1.8
org.glassfish.hk2 hk2-api 2.4.0-b34
org.glassfish.hk2 hk2-locator 2.4.0-b34
org.glassfish.hk2 hk2-utils 2.4.0-b34
org.glassfish.hk2 osgi-resource-locator 1.0.1
org.glassfish.hk2.external aopalliance-repackaged 2.4.0-b34
org.glassfish.hk2.external javax.inject 2.4.0-b34
org.glassfish.jersey.bundles.repackaged jersey-guava 2.22.2
org.glassfish.jersey.containers jersey-container-servlet 2.22.2
org.glassfish.jersey.containers jersey-container-servlet-core 2.22.2
org.glassfish.jersey.core jersey-client 2.22.2
org.glassfish.jersey.core jersey-common 2.22.2
org.glassfish.jersey.core jersey-server 2.22.2
org.glassfish.jersey.media jersey-media-jaxb 2.22.2
org.javassist javassist 3.18.1-GA
org.json4s json4s-ast_2.12 3.5.3
org.json4s json4s-core_2.12 3.5.3
org.json4s json4s-jackson_2.12 3.5.3
org.json4s json4s-scalap_2.12 3.5.3
org.jvnet tiger-types 1.4
org.lz4 lz4-java 1.4.0
org.mortbay.jetty jetty 6.1.26
org.mortbay.jetty jetty-sslengine 6.1.26
org.mortbay.jetty jetty-util 6.1.26
org.objenesis objenesis 2.5.1
org.roaringbitmap RoaringBitmap 0.7.45
org.roaringbitmap shims 0.7.45
org.scala-lang scala-compiler 2.12.10
org.scala-lang scala-library 2.12.10
org.scala-lang scala-reflect 2.12.10
org.scala-lang.modules scala-parser-combinators_2.12 1.1.0
org.scala-lang.modules scala-xml_2.12 1.0.5
org.scalanlp breeze-macros_2.12 0.13.2
org.scalanlp breeze_2.12 0.13.2
org.slf4j jcl-over-slf4j 1.7.16
org.slf4j jul-to-slf4j 1.7.16
org.slf4j log4j-over-slf4j 1.7.30
org.slf4j slf4j-api 1.7.16
org.spire-math spire-macros_2.12 0.13.0
org.spire-math spire_2.12 0.13.0
org.tukaani xz 1.5
org.typelevel machinist_2.12 0.6.1
org.typelevel macro-compat_2.12 1.1.1
org.xerial.snappy snappy-java 1.1.7.5
org.yaml snakeyaml 1.15
oro oro 2.0.8
xerces xercesImpl 2.9.1
xmlenc xmlenc 0.52

Environment for Stream Processing 5.0

Group ID Artifact ID Version
ch.qos.logback logback-classic 1.2.3
ch.qos.logback logback-core 1.2.3
com.esotericsoftware.kryo kryo 2.24.0
com.esotericsoftware.minlog minlog 1.2
com.esotericsoftware.reflectasm reflectasm 1.09
org.apache.calcite calcite-core 1.26.0
org.apache.calcite calcite-linq4j 1.26.0
org.apache.calcite.avatica avatica-core 1.17.0
org.apache.commons commons-compress 1.21
org.apache.commons commons-lang3 3.3.2
org.apache.commons commons-math3 3.5
org.apache.flink flink-annotations 1.13.5
org.apache.flink flink-cep_2.12 1.13.5
org.apache.flink flink-clients_2.12 1.13.5
org.apache.flink flink-connector-base 1.13.5
org.apache.flink flink-connector-files 1.13.5
org.apache.flink flink-container_2.12 1.13.5
org.apache.flink flink-core 1.13.5
org.apache.flink flink-csv 1.13.5
org.apache.flink flink-file-sink-common 1.13.5
org.apache.flink flink-hadoop-fs 1.13.5
org.apache.flink flink-java 1.13.5
org.apache.flink flink-json 1.13.5
org.apache.flink flink-kubernetes_2.12 1.13.5
org.apache.flink flink-mapr-fs 1.13.5
org.apache.flink flink-mesos_2.12 1.13.5
org.apache.flink flink-metrics-core 1.13.5
org.apache.flink flink-optimizer_2.12 1.13.5
org.apache.flink flink-queryable-state-client-java 1.13.5
org.apache.flink flink-runtime-web_2.12 1.13.5
org.apache.flink flink-runtime_2.12 1.13.5
org.apache.flink flink-scala_2.12 1.13.5
org.apache.flink flink-shaded-asm-7 7.1-13.0
org.apache.flink flink-shaded-guava 18.0-13.0
org.apache.flink flink-shaded-jackson 2.12.1-13.0
org.apache.flink flink-shaded-netty 4.1.49.Final-13.0
org.apache.flink flink-shaded-zookeeper-3 3.4.14-13.0
org.apache.flink flink-sql-parser 1.13.5
org.apache.flink flink-sql-parser-hive 1.13.5
org.apache.flink flink-statebackend-rocksdb_2.12 1.13.5
org.apache.flink flink-streaming-java_2.12 1.13.5
org.apache.flink flink-streaming-scala_2.12 1.13.5
org.apache.flink flink-table-api-java 1.13.5
org.apache.flink flink-table-api-java-bridge_2.12 1.13.5
org.apache.flink flink-table-api-scala-bridge_2.12 1.13.5
org.apache.flink flink-table-api-scala_2.12 1.13.5
org.apache.flink flink-table-common 1.13.5
org.apache.flink flink-table-planner-blink_2.12 1.13.5
org.apache.flink flink-table-planner_2.12 1.13.5
org.apache.flink flink-table-runtime-blink_2.12 1.13.5
org.apache.flink flink-yarn_2.12 1.13.5
org.apache.flink force-shading 1.13.5
org.apache.mesos mesos 1.7.0
org.codehaus.janino commons-compiler 3.0.11
org.codehaus.janino janino 3.0.11
org.javassist javassist 3.24.0-GA
org.objenesis objenesis 2.1
org.reactivestreams reactive-streams 1.0.2
org.slf4j log4j-over-slf4j 1.7.25
org.slf4j slf4j-api 1.7.15

results matching ""

    No results matching ""