You can use Apache Flink on Amazon EMR for unified BATCH and STREAM processing of Apache Hive Tables or metadata of any Flink tablesource such as Iceberg, Kinesis or Kafka. For more information, see Fault-tolerant execution in Trino. Fault-tolerant execution mitigates query failures by retrying failed queries or their component tasks. To support long running queries, Trino now includes a fault-tolerant execution mechanism. For more information, see Configure cluster logging and debugging. The new capability ensures that log files generated on the cluster persist on Amazon S3 even after the node is terminated. Previously, you could only archive log files to Amazon S3 during cluster termination. For more information, see Using Amazon Redshift integration for Apache Spark with Amazon EMR.Īmazon EMR release 6.9.0 adds support for archiving logs to Amazon S3 during cluster scale-down. Previously an open-source tool, the native integration is a Spark connector that you can use to build Apache Spark applications that read from and write to data in Amazon Redshift and Amazon Redshift Serverless. The Amazon Redshift integration for Apache Spark is included in Amazon EMR releases 6.9.0 and later. Amazon EMR release 6.9.0 supports Apache Spark RAPIDS 22.08.0, Apache Hudi 0.12.1, Apache Iceberg 0.14.1, Trino 398, and Tez 0.10.2.Īmazon EMR release 6.9.0 includes a new open-source application, Delta Lake 2.1.0.
0 Comments
Leave a Reply. |