Logo

dev-resources.site

for different kinds of informations.

Parse a csv file where field values are enclosed by quotation marks and contain carriage return #eg35

Published at
9/2/2024
Categories
sql
development
programming
esproc
Author
esproc_spl
Author
10 person written this
esproc_spl
open
Parse a csv file where field values are enclosed by quotation marks and contain carriage return #eg35

Here is a csv file. Its 3rd column contains strings of a special format – they are enclosed by double quotation marks and some occupy multiple rows (which means they contain carriage return).

Image description
Task: Use Java to reorganize the csv file as a standard two-dimensional table.

Image description
Write the following SPL statement:

=file(""data.csv"").import@tcoq().run(description=replace(description,""\n"",""""))

import() function parses the csv file as a two dimensional table; @t option enables importing the 1st row as column names, @c option enables using commas as the separator, @o option handles the unescaped carriage return in lines, and @q option enables removing quotation marks at both sides before parsing. run() function modifies the two dimensional table and return the modified table.

Read How to Call a SPL Script in Java to find how to integrate SPL into a Java application.

This is one of the problems on StackOverflow. You can click on it to see that the conventional solution is quite complicated, but the SPL approach is really simple and efficient.

SPL open source address

esproc Article's
30 articles in total
Favicon
Add records that meet the criteria before each group after grouping :From SQL to SPL
Favicon
Multi combination condition grouping and aggregation #eg93
Favicon
Split a Huge CSV File into Multiple Smaller CSV Files #eg69
Favicon
Group & Summarize a CSV File #eg68
Favicon
Getting positions of members according to primary key values #eg58
Favicon
Getting members according to primary key values #eg63
Favicon
How to Access Databases using One SQL Statement #eg71
Favicon
Filter a CSV file and re-arrange it by category #eg60
Favicon
Getting positions of members based on a specified condition #eg46
Favicon
Convert Each Whites-space-separated Text Block into a Row #eg62
Favicon
Perform Distinct on Ordered Numbers in a Text File #eg61
Favicon
Parse a csv file having a primary-sub tables structure #eg41
Favicon
Convert CSV Data into Multilevel JSON #eg56
Favicon
Add a compute column to a csv file #eg40
Favicon
SQL, in each group modify the null value of a specified column as its neighboring value #eg43
Favicon
Get the whole group where at least one member meets the specified condition #eg36
Favicon
Parse a csv file where field values are enclosed by quotation marks and contain carriage return #eg35
Favicon
Replace Duplicate Digits in Every 9-digit Number in a Text File with Non-duplicate Ones #eg52
Favicon
Reverse Rows in a Text File #eg51
Favicon
The Difference between Each Value in a Certain Column and Its Previous One and Display Result
Favicon
Java, perform COUNT on each group of a large csv file #eg33
Favicon
SQL, extract unique values of JSON format field from each group #eg42
Favicon
Multi-condition filtering #eg48
Favicon
Getting members based on a specified condition #47
Favicon
Read specified columns from a csv file #eg44
Favicon
Something could double the development efficiency of Java programmers
Favicon
Java, fill each row having a null value in a csv file with values in the directly previous row #eg32
Favicon
To Index Data is To Sort Data
Favicon
Clear duplicate lines and lines having missing values from a csv file #eg24
Favicon
SQL, Set different flags for different groups according to whether there are duplicate values #eg19

Featured ones: