[Node.js]Convert CSV file to JSON file

A request came in to implement a web application with a lot of article data and drawing in the same way,
The requirement for multilingualisation (ja/en) came in along with the request.

It is difficult to do this manually one by one, so we created a simple tool to convert from CSV to JSON.

1. liaise with the person creating the content and decide on the format (Google Spreadsheet).
2. convert the format to CSV
   3.Pass CSV to the tool and convert it to JSON data that is assumed to be rendered by Next.js.

By doing this, data conversion is automated, allowing you to concentrate on implementation.

As a prerequisite, implementation is carried out in an environment where Node.js runs.

The final product to be created

CSV files (. /data/data.csv) is read and converted to a JSON file.

This is supposed to be a tool to create JSON data to be completed only in the front end [Next.js].

The CSV header is fixed and is assumed to be “写真,記事日本語,記事英語,緯度経度,時代タグ,建築タグ”.

By executing (node main.js ), various JSON files processed for Next.js are created.

data.json (used to map and draw data)
ja.json (used as i18n language files)
en.json (used as i18n language file)

Library installation

There was a library to convert csv files to json, so we will use it.

npm i --save csvtojson

Create functions to convert

It has constants for the location of the CSV file and the location of the generated JSON file, and calls the csv2json function to automatically convert the CSV to JSON.

The csvtojson library is used in line 10 to convert CSV to JSON.

The JSON file processed for i18n is generated at line 17.

A summarised JSON file is generated at line 28.

const fs = require("fs");
const csv = require("csvtojson");

const inputCsvPath = "./data/data.csv";
const outputPath = "./out";
const lang = ["ja", "en"];
const jsonIndent = 2;

exports.csv2json = async function () {
  const jsonArray = await csv().fromFile(inputCsvPath);

  if (!fs.existsSync(outputPath)) {
    fs.mkdirSync(outputPath, { recursive: true });
  }

  // Create Lang files
  for (const locale of lang) {
    const langJson = { article: {} };
    langJson.article = jsonArray.map((d, index) => {
      return { [`trans_${index}`]: locale == "ja" ? d["記事日本語"] : d["記事英語"] };
    });

    const jsonStr = JSON.stringify(langJson, undefined, jsonIndent);
    fs.writeFileSync(`${outputPath}/${locale}.json`, jsonStr);
  }

  // data
  const additionalJson = jsonArray.map((d, index) => {
    return {
      [`photoName`]: d["写真"],
      transKey: `trans_${index}`,
      [`latLng`]: d["緯度経度"].replace(/\s+/g, ""),
      [`tagPeriod`]: d["時代タグ"],
      [`tagArchitecture `]: d["建築タグ"],
    };
  });

  const jsonStr = JSON.stringify(additionalJson, undefined, jsonIndent);
  fs.writeFileSync(`${outputPath}/data.json`, jsonStr);
};

Creating the execution part

You could call it directly, but we wanted to separate the roles, so we use the main.js file as the endpoint.

The main function in main.js calls convert.js csv2json and executes it.

const app = require("./convert");

main();

async function main() {
  const startTime = Date.now();

  await app.csv2json();

  const endTime = Date.now();

  console.log("completed !!");
  console.log(`process time : ${endTime - startTime} ms`);
}

GitHub

If you want to see the whole code, go ahead.

https://github.com/potatowithbutter1987/ConvertCsvToJson

Summary

It would be a good idea to check out the libraries first, use them if they are available, and if not, create your own.
Again, thanks to the use of a library to convert CSV to JSON, I was able to focus on the subsequent processing.

Set the rules and send the automation to the engineer.