Logo

dev-resources.site

for different kinds of informations.

Add records that meet the criteria before each group after grouping :From SQL to SPL

Published at
12/27/2024
Categories
sql
spl
esproc
Author
esproc_spl
Categories
3 categories in total
sql
open
spl
open
esproc
open
Author
10 person written this
esproc_spl
open
Add records that meet the criteria before each group after grouping :From SQL to SPL

In a certain view of the PostgreSQL database, the row_index field is an underscore separated string and also a grouping field. Some groups' row_index can be divided into 3 parts, while others can be divided into 2 parts.

Image description

Now, before each group of records where row_index can be divided into 3 parts, add the group of records where row_index can be divided into 2 parts, and modify row_index to the row_index of each group.

Image description
The records where row_index can be divided into three parts can be grouped by row_index, and then each group of records can be processed by merging the records where row_index can be divided into two parts before each group of records. But after SQL grouping, it must aggregate immediately, and subsets cannot be kept to continue processing each group of records. This requires a detour to solve, using multi-layer nested window functions to bypass this problem, which is difficult to code.

SPL supports retaining subsets after grouping, allowing for continued processing of each group of records.

Image description

A1: Query the database through JDBC.

A2: Retrieve the records where row_index can be divided into two parts.

A3: Remove A2 from the complete data, which means getting records that row_index can be divided into three parts. Group these records while keeping the order unchanged. The symbol \ represents the difference set, and group@u indicates keeping the original order after grouping.

A4: Loop each group of data: Create a new two-dimensional table according to A2, change row_index to the row_index of this group, keep other fields unchanged, and merge the new two-dimensional table with the data of this group. Finally, merge the data of each group. The symbol | represents merging set members.

Open source SPL source address

esproc Article's
30 articles in total
Favicon
Add records that meet the criteria before each group after grouping :From SQL to SPL
Favicon
Multi combination condition grouping and aggregation #eg93
Favicon
Split a Huge CSV File into Multiple Smaller CSV Files #eg69
Favicon
Group & Summarize a CSV File #eg68
Favicon
Getting positions of members according to primary key values #eg58
Favicon
Getting members according to primary key values #eg63
Favicon
How to Access Databases using One SQL Statement #eg71
Favicon
Filter a CSV file and re-arrange it by category #eg60
Favicon
Getting positions of members based on a specified condition #eg46
Favicon
Convert Each Whites-space-separated Text Block into a Row #eg62
Favicon
Perform Distinct on Ordered Numbers in a Text File #eg61
Favicon
Parse a csv file having a primary-sub tables structure #eg41
Favicon
Convert CSV Data into Multilevel JSON #eg56
Favicon
Add a compute column to a csv file #eg40
Favicon
SQL, in each group modify the null value of a specified column as its neighboring value #eg43
Favicon
Get the whole group where at least one member meets the specified condition #eg36
Favicon
Parse a csv file where field values are enclosed by quotation marks and contain carriage return #eg35
Favicon
Replace Duplicate Digits in Every 9-digit Number in a Text File with Non-duplicate Ones #eg52
Favicon
Reverse Rows in a Text File #eg51
Favicon
The Difference between Each Value in a Certain Column and Its Previous One and Display Result
Favicon
Java, perform COUNT on each group of a large csv file #eg33
Favicon
SQL, extract unique values of JSON format field from each group #eg42
Favicon
Multi-condition filtering #eg48
Favicon
Getting members based on a specified condition #47
Favicon
Read specified columns from a csv file #eg44
Favicon
Something could double the development efficiency of Java programmers
Favicon
Java, fill each row having a null value in a csv file with values in the directly previous row #eg32
Favicon
To Index Data is To Sort Data
Favicon
Clear duplicate lines and lines having missing values from a csv file #eg24
Favicon
SQL, Set different flags for different groups according to whether there are duplicate values #eg19

Featured ones: