Table In Flink

Liao Jiayi Liao Jiayi #Flink#Apache Flink

Some notes on the Table APIs, which I had not used before.

Translated from Chinese with AI · Read the original

I had not used the Table APIs before, so these are a few notes.

Structure of Program

A program using the Table API is structured roughly as follows:

// create a TableEnvironment for specific planner batch or streaming
TableEnvironment tableEnv = ...; // see "Create a TableEnvironment" section


// register a Table
tableEnv.registerTable("table1", ...)            // or
tableEnv.registerTableSource("table2", ...);     // or
// register an output Table
tableEnv.registerTableSink("outputTable", ...);


// create a Table from a Table API query
Table tapiResult = tableEnv.scan("table1").select(...);
// create a Table from a SQL query
Table sqlResult  = tableEnv.sqlQuery("SELECT ... FROM table2 ... ");


// emit a Table API result Table to a TableSink, same for SQL result
tapiResult.insertInto("outputTable");


// execute
tableEnv.execute("java_job");

TableEnvironment has Batch and Stream variants. Their bridge modules provide conversions to and from DataSet and DataStream, respectively.

There are three ways to register a Table:

  • registerTable // register with Table
  • registerTableSource // register with TableSource
  • registerTableSink // register with TableSink

Registration adds the corresponding table to the Catalog. Hive and InMemory modes are currently supported.