Logo

dev-resources.site

for different kinds of informations.

Java, perform COUNT on each group of a large csv file #eg33

Published at
8/28/2024
Categories
sql
development
database
esproc
Author
esproc_spl
Categories
4 categories in total
sql
open
development
open
database
open
esproc
open
Author
10 person written this
esproc_spl
open
Java, perform COUNT on each group of a large csv file #eg33

data.csv is a large csv file that cannot fit into the memory; its 3rd column is the grouping column, as shown below:

Date,Time,Sub User,Access Method

10-10-2023,00:03:06,JL,cli

10-10-2023,00:02:20,TW2JL,app

10-10-2023,00:03:26,JL,cli

10-10-2023,00:03:34,JL,cli

10-10-2023,00:03:35,JL,cli

10-10-2023,00:03:46,JL,cli

10-10-2023,00:04:09,JL,cli

10-10-2023,00:04:51,JL,cli

10-10-2023,00:04:56,JL,cli

10-10-2023,00:05:58,JL,cli

10-10-2023,00:06:29,JL,cli

10-10-2023,00:06:42,JL,cli

10-10-2023,00:26:35,TW2JL,app

10-10-2023,00:30:01,TW2JL,app

10-10-2023,00:30:02,TW2JL,app

10-10-2023,00:30:05,TW2JL,app

10-10-2023,00:33:42,TW2JL,app

10-10-2023,00:36:36,TW2JL,app

10-10-2023,00:45:10,TW2JL,app

10-10-2023,00:53:01,TW2JL,app

10-10-2023,00:53:24,TW2JL,app

10-10-2023,01:03:14,TW2JL,app

10-10-2023,01:03:18,TW2JL,app

10-10-2023,01:03:20,TW2JL,app

Task: Use Java to group values in the 3rd column and count record in each group. Below is the expected result:

Sub User cnt

JL 11

TW2JL 13

Write the following SPL statement:

=T@c(""data.csv"").groups("'Sub User"';count(1):cnt)

T()function parses the csv file; @c option enables using the cursor mode. groups() function performs grouping and aggregation.

Read How to Call a SPL Script in Java to find how to integrate SPL into a Java application.

Source

SPL open source address

esproc Article's
30 articles in total
Favicon
Add records that meet the criteria before each group after grouping :From SQL to SPL
Favicon
Multi combination condition grouping and aggregation #eg93
Favicon
Split a Huge CSV File into Multiple Smaller CSV Files #eg69
Favicon
Group & Summarize a CSV File #eg68
Favicon
Getting positions of members according to primary key values #eg58
Favicon
Getting members according to primary key values #eg63
Favicon
How to Access Databases using One SQL Statement #eg71
Favicon
Filter a CSV file and re-arrange it by category #eg60
Favicon
Getting positions of members based on a specified condition #eg46
Favicon
Convert Each Whites-space-separated Text Block into a Row #eg62
Favicon
Perform Distinct on Ordered Numbers in a Text File #eg61
Favicon
Parse a csv file having a primary-sub tables structure #eg41
Favicon
Convert CSV Data into Multilevel JSON #eg56
Favicon
Add a compute column to a csv file #eg40
Favicon
SQL, in each group modify the null value of a specified column as its neighboring value #eg43
Favicon
Get the whole group where at least one member meets the specified condition #eg36
Favicon
Parse a csv file where field values are enclosed by quotation marks and contain carriage return #eg35
Favicon
Replace Duplicate Digits in Every 9-digit Number in a Text File with Non-duplicate Ones #eg52
Favicon
Reverse Rows in a Text File #eg51
Favicon
The Difference between Each Value in a Certain Column and Its Previous One and Display Result
Favicon
Java, perform COUNT on each group of a large csv file #eg33
Favicon
SQL, extract unique values of JSON format field from each group #eg42
Favicon
Multi-condition filtering #eg48
Favicon
Getting members based on a specified condition #47
Favicon
Read specified columns from a csv file #eg44
Favicon
Something could double the development efficiency of Java programmers
Favicon
Java, fill each row having a null value in a csv file with values in the directly previous row #eg32
Favicon
To Index Data is To Sort Data
Favicon
Clear duplicate lines and lines having missing values from a csv file #eg24
Favicon
SQL, Set different flags for different groups according to whether there are duplicate values #eg19

Featured ones: