Query Engine
The Query Engine service lets you run SQL queries directly on data stored in Object Storage, without needing to create or maintain a separate analytical database. This service is built on Trino and is created exclusively in your account in the Cloud Container. The service connects to data through the Data Catalog, so there's no need for ETL or moving data.
In many organizations, analytical data is kept in Object Storage as files such as Parquet or CSV. This storage method has advantages, but running analytical queries on this data without suitable infrastructure can come with the following problems:
- Running analytical queries requires creating or maintaining a separate analytical database
- Moving data to analytical engines is time-consuming and costly
- Data science teams and analysts need self-service access
- Direct access to raw data can create a security risk
As a result, organizations need a tool that makes running analytical queries possible without copying data or ETL, and with access control, so analysis and secure access to data are provided at the same time.
Features
The Query Engine is designed to provide real-time analysis, connections to analytics and BI tools, and controlled access to data, without data teams needing to build and manage separate infrastructure.
The Query Engine's main features include:
- Running queries directly on Object Storage: No ETL, no data movement
- Iceberg support: Features such as snapshots, time travel, partition evolution, and more
- Standard and compatible: Trino CLI, JDBC, ODBC, and BI tools
- Integrated with the Data Catalog: Automatic discovery of stored tables and schemas
- High security: Each engine is created and managed in the user's account
With these features, in analytics scenarios users can connect their BI tools directly to the Query Engine and build analytical dashboards on stored data. Data science teams can also run ad-hoc analyses without copying data, using only the resources defined for the engine.
Setting Up a Query Engine
To use the Query Engine, you first need to create a Data Catalog in Arvancloud Object Storage. The Data Catalog guide helps you with this.
After creating the data catalog, go to Query Engines in the Object Storage menu.
Click Create Query Engine, and on the page that opens, enter the following details:
-
Query engine name: Choose a name containing lowercase English letters, numbers, and -.
-
Compute resources: Specify the amount of processor and RAM you need.
Based on the service's requirements, the disk and ephemeral storage sizes are set by the system and can't be changed.
These resources are created in your account, in the Cloud Container section.
Note that the resource limits for creating an engine are based on your account. If you need more resources, submit an increase request in Quota Management in the profile settings.
-
Data catalog: In this section, select a data catalog you've already created to connect to the query engine. You can also create a new data catalog in this section.
-
Connection settings: A root username is created for you by default. Enter a 10-character password for it.
Finally, click Create Query Engine.
After creation is complete, click the query engine's name and take the values needed to configure your client from the Connection Details tab.
Depending on your needs and the resources used, you can also connect more data catalogs to one engine and manage different data.
To connect to the Query Engine with a username and password, the connection must use TLS. When the engine is created, a self-signed certificate is created on the host by default; however, since this certificate isn't valid for some clients, for initial testing you usually need to use options such as --insecure.
For a secure connection in production, it is recommended that you create a DNS record on your organization's domain, set its value to the engine's IP, and then enforce access only over HTTPS. In this case, clients can connect to the service using the domain name, and the connection is valid and secure.
The guides to adding DNS records and HTTPS settings help you with this.
Connecting to the Query Engine
Several ways of connecting to the Query Engine in Arvancloud Object Storage are supported:
- Trino CLI: With full support for running SQL commands
- JDBC / ODBC: Suitable for BI tools, including Power BI, Tableau, Superset, and DBeaver
- Golang SDK: For developing services and notebooks
Connecting via JDBC
Suitable for Power BI, Tableau, and Superset
Sample URL:
jdbc:trino://<engine-host>:443/catalog/<catalog>/schema/<schema>;
user=<username>;
password=<password>;
SSL=true;
Connecting via ODBC
Suitable for Power BI Desktop and Excel
Sample settings:
Driver: Trino ODBC Driver
Host: <engine-host>
Port: 443
Authentication: Username + password
SSL: Enabled
Connecting via the Golang SDK
Sample code:
import (
"context"
"crypto/tls"
"database/sql"
"fmt"
"log"
"net/http"
"strings"
"time"
"github.com/trinodb/trino-go-client/trino"
_ "github.com/trinodb/trino-go-client/trino"
)
func main() {
// Trino connection configuration
host := "Trino-server-address"
username := "Username" // Replace with your Trino username
password := "Password" // Replace with your Trino password
catalogName := "Your-catalog-name"
tableName := "myTable"
schemaName := "mySchema"
fullTablePath := fmt.Sprintf("%v.%v.%v", catalogName, schemaName, tableName)
// Create a Config struct
customClient := &http.Client{
Transport: &http.Transport{
TLSClientConfig: &tls.Config{
InsecureSkipVerify: true, // WARNING: Skips server cert verification (dev only)
MinVersion: tls.VersionTLS13,
},
},
}
// Register the custom client with the Trino driver
trino.RegisterCustomClient("trino_client", customClient)
// Configure Trino connection
config := &trino.Config{
ServerURI: fmt.Sprintf("https://%v:%v@%v", username, password, host),
Catalog: catalogName,
Source: "go-client",
CustomClientName: "trino_client", // Reference the custom client
}
// Format the DSN
dsn, err := config.FormatDSN()
if err != nil {
log.Fatalf("Failed to format DSN: %v", err)
}
// Open a connection to Trino
db, err := sql.Open("trino", dsn)
if err != nil {
log.Fatalf("Failed to open connection: %v", err)
}
defer db.Close()
ctx := context.Background()
// Test the connection
err = db.PingContext(ctx)
if err != nil {
log.Fatalf("Failed to connect to Trino server: %v", err)
}
fmt.Println("Successfully connected to Trino server")
// Example query to verify the catalog
query := "SHOW CATALOGS"
rows, err := db.QueryContext(ctx, query)
if err != nil {
log.Fatalf("Failed to query catalogs: %v", err)
}
defer rows.Close()
// Print the list of catalogs
fmt.Println("Available catalogs:")
for rows.Next() {
var catalog string
if err := rows.Scan(&catalog); err != nil {
log.Fatalf("Failed to scan row: %v", err)
}
fmt.Println(catalog)
}
if err := rows.Err(); err != nil {
log.Fatalf("Error iterating rows: %v", err)
}
// Execute SQL statements one by one
sqlStatements := []string{
fmt.Sprintf("SHOW SCHEMAS FROM %s", catalogName),
fmt.Sprintf("CREATE SCHEMA %s.%s", catalogName, schemaName),
fmt.Sprintf("CREATE TABLE %s ( id INTEGER, name VARCHAR, created_at TIMESTAMP )", fullTablePath),
fmt.Sprintf("INSERT INTO %s (id, name, created_at) VALUES (1, 'John Doe', CAST('2025-04-26 09:06:04' AS TIMESTAMP))", fullTablePath),
fmt.Sprintf("SELECT * FROM %s LIMIT 10", fullTablePath),
}
for _, sql := range sqlStatements {
rows, err := db.QueryContext(ctx, sql)
if err != nil {
log.Fatalf("Failed to execute query %q: %v", sql, err)
}
defer rows.Close()
// Print results for SELECT query
if strings.HasPrefix(strings.TrimSpace(sql), "SELECT") {
fmt.Println("\nQuery results:")
for rows.Next() {
var id int
var name string
var createdAt time.Time
if err := rows.Scan(&id, &name, &createdAt); err != nil {
log.Fatalf("Failed to scan row: %v", err)
}
fmt.Printf("ID: %d, Name: %s, Created At: %v\n", id, name, createdAt)
}
}
if err := rows.Err(); err != nil {
log.Fatalf("Error iterating rows: %v", err)
}
}
}
Connecting with DBeaver
DBeaver is a graphical tool for connecting to databases and running queries that, besides traditional databases such as PostgreSQL and MySQL, also supports JDBC-compatible analytical engines and Trino. With this tool, you can run queries and see the results without needing the command line.
After installing the latest version of this software, click New Database Connection and select Trino from the list.
Then enter your connection details.
- Set Host to the engine's domain or IP.
- Enter 443 for the port.
If you aren't yet using a domain with a valid certificate, you need to set SSLVerification to NONE in Driver Properties for the connection to be established.
Before finalizing the settings, click Test Connection to check the connection.
The query environment is now ready to use.