Skip to main content

TPC-DS Benchmarking

TPC-DS is a decision support benchmark developed by the Transaction Processing Performance Council (TPC). It uses more comprehensive test datasets and complex SQL queries than TPC-H.

TPC-DS models several generally applicable aspects of a decision support system, including queries and data maintenance. TPC-DS aims to provide a comprehensive and realistic workload for testing and evaluating the performance of database systems in a retail environment. The TPC-DS benchmark simulates the sales and return data of three sales channels (stores, Internet, and catalog) in a retail enterprise. In addition to creating tables for sales and return data models, it also includes a simple inventory system and a promotion system.

This benchmark tests a total of 99 complex SQL queries against 24 tables whose data size ranges from 1 GB to 3 TB. The main performance metric is the response time of each query, which is the duration between the time a query is submitted to the time the result is returned.

Test Conclusions​

The test is performed against the TPC-DS 1 TB dataset on Cel er Data and Trino. The unit of the following results are milliseconds.

TPC-DS-CD

For CelerData, the test queries are performed on its native tables (under both classic and elastic architecture) and Iceberg Catalog (under elastic architecture). For Trino, the test queries are performed on the same Iceberg Catalog. Both tests for CelerData and Trino on the Iceberg Catalog use AWS Glue as the metastore, and Parquet-formatted data with ZSTD compression. For Databricks Photon, the test queries are performed on Delta Lake tables. It also uses AWS Glue as the metastore, and Parquet-formatted data with ZSTD compression.

The test results are:

  • Queries performed on the OLAP table in the CelerData classic cluster took 295 seconds.
  • Queries performed on the OLAP table in the CelerData elastic cluster took 299 seconds.
  • Queries performed on Iceberg Catalog in the CelerData elastic cluster took 311 seconds.
  • Queries performed on Iceberg Catalog in Trino took 2552 seconds.
  • Queries performed on Delta Lake in Databricks Photon took 415 seconds.

The conclusions are:

  • CelerData delivers comparable performance in both classic and elastic architecture

    In OLAP table query scenarios, CelerData in the classic mode achieves 1.01× the performance of the elastic mode, indicating minimal performance overhead from storage-compute separation.

  • CelerData significantly outperforms Trino and Databricks Photon in Lakehouse query scenarios out of the box, with even greater advantages in OLAP table queries

    • In Iceberg Catalog query scenarios, CelerData delivers 8.2× the performance of Trino and 1.3x the performance of Databricks Photon.
    • In OLAP table queries, the performance gap widens further—CelerData achieves 8.52× (under classic mode) and 8.54× (under elastic mode) the performance of Trino, and 1.4x the performance of Databricks Photon (under both classic and elastic mode).

Test Preparation​

Hardware​

CelerData and Trino are deployed on hosts of the same configurations - AWS m6id.4xlarge.

Spec
Instance Number5
vCPU16
Memory (GiB)64
Instance Storage (GB)1x950 NVMe SSD
Network Bandwidth (Gbps)Up to 12.5
EBS Bandwidth (Gbps)Up to 10

Software​

CelerDataTrinoDatabricks Photon
Cluster SizeOne FE, four BEOne Coordinator, four WorkersOne Driver, four Workers
Version3.5.047516.4 LTS
Release Date2025.6.132025.6.52025.5.9
Configurationset query_mem_limit=66571993088; (62 GB)query.max-memory=224GB
query.max-memory-per-node=35GB		Default

Test results​

The following table shows the performance test results on 99 queries. The unit of query latency is milliseconds. All queries are warmed up 1 time and then executed 3 times to take the average value as the result. Trino vs CelerData on Iceberg and Databricks vs CelerData in the table header means using the query response time of Trino to divide the query response time of CelerData. A larger value indicates better performance of CelerData. Before the test on Iceberg Catalog, ANALYZE TABLE and EXPLAIN COST statements are executed on CelerData to collect statistics.

QueryCelerDataTrinoDatabricks PhotonTrino vs CelerData On IcebergDatabricks vs CelerData
OLAP TableIceberg CatalogIceberg CatalogDelta Lake
ClassicElasticElastic
SUM29489529921931068125520764158098.211.34
Q01748605709227618913.212.67
Q021136934173321289374512.282.16
Q03193146404176410024.372.48
Q041818215935159382002491489512.560.93
Q054655411748557417143.190.98
Q06209176336290911028.663.28
Q076669411597363820722.281.30
Q08201175331253412397.663.74
Q094549397055183612567206.551.22
Q10295308533247114154.642.65
Q1110888966999161332301130213.441.14
Q1212412420710676675.153.22
Q13418444180518297256710.141.42
Q14159681473515007143172151569.541.01
Q1553345655521589923.891.79
Q1653154495014035194114.772.04
Q1710129421638548728953.351.77
Q188619311678473623802.821.42
Q19234264502187023413.734.66
Q2015216623712578305.303.50
Q2186801999456184.753.11
Q22181018232185787124543.601.12
Q23831207942851596488653341939.470.66
Q24762090351150943436235533.772.05
Q258449191389434724793.131.78
Q26376502762296112773.891.68
Q274585331333421919253.171.44
Q2831703222461924678118185.342.56
Q291324209621491064071654.953.33
Q30294303501277013365.532.67
Q312193203024071026035604.261.48
Q3211121928110357003.682.49
Q33237241613274312794.472.09
Q34375416660245526433.724.00
Q35139214051600802323005.011.44
Q365435281076334217923.111.67
Q37253242208617896729.704.65
Q384706454652143840849787.370.95
Q39361348813569636817.014.53
Q40122131515155412633.022.45
Q414747752714273.615.69
Q42811011949965455.132.81
Q43488674653410816246.292.49
Q441450157450031646837703.290.75
Q4541840952922129084.181.72
Q46146217041502478025943.181.73
Q4727972759324242704543513.171.68
Q4849152313351246418069.341.35
Q494965141344382719272.851.43
Q503738496841872177148625.201.16
Q514725457434181224746033.581.35
Q529110622410075924.502.64
Q53457628894176812011.981.34
Q5420825777731864220541.012.84
Q558710521911875805.422.65
Q56175185500213312724.272.54
Q5717591728213226042396312.211.86
Q58239251522303210505.812.01
Q593745373633612081932896.190.98
Q60254295649274914324.242.21
Q61332338936205419262.192.06
Q62509528902510412505.661.39
Q63447629866171911621.981.34
Q6444458719829116990119672.051.44
Q652836277136392061668945.671.89
Q663983881094351120053.211.83
Q6729203283922168489430383994.121.77
Q68255331708288320184.072.85
Q69276281468218111904.662.54
Q702815291734542149941956.221.21
Q712781110639257885624.0313.40
Q72163618745077504573526599.381.04
Q7319523037719339425.132.50
Q749086898690827807178538.600.86
Q7562587410899226270100892.921.12
Q762466232948241126645502.340.94
Q77240237696278511674.001.68
Q7816750158601923151497167392.680.87
Q79214223581680557220053.321.19
Q805618502534476227741.881.09
Q81493500763404020425.292.68
Q8261160583911622126213.851.50
Q83160212400227413885.693.47
Q8421344332030868989.642.81
Q854736851241542417904.371.44
Q86733738856342210724.001.25
Q874651488951434130058428.031.14
Q88156515862821307381389210.904.92
Q89534738916222714202.431.55
Q90326336634389014426.142.27
Q911029823220358148.773.51
Q92851432509946533.982.61
Q933790460357762292637553.970.65
Q94599984995588415825.911.59
Q9517593003266629607205311.110.77
Q96151014092561490521031.920.82
Q974745501355172272658674.121.06
Q98285313350191136905.4610.54
Q991165119214961097023607.331.58