CSV Data Into App + Search

Hi! Thank you for taking the time to read this. I am relatively new to Swift and am trying to create an app that will self-download/self-update from a .csv file from a URL, which will be filtered by a search from the user. So, the user would search a keyword of a row within the .csv file i.e. “Oregon” and the data for Oregon’s row would pop up. I am not sure how to make the data from the URL self-updating (like a network link) nor how to create the search through the CSV data. Is there a video from CodeWithChris or an article/video that would address this? Sorry, I am very new and am trying to figure out how to start :slight_smile: ! If any question could be clarified, I would really appreciate it as I know I am asking for you to take your time to address my concerns. Thank you for reading this again. Have a great day to you! :innocent:

I don’t know of any video that might cover what you are looking for but importing the file is relatively straight forward even if it is in a remote place requiring a URLSession to get it.

The first step is to break the file up into an array of strings with each entry in the array equating to a line from the CSV file. Each line can be separated by identifying the line terminating character \n.

Lets say that in your URLSession the file you want is located in a fictitious place like here “https://someplace.com/data.csv” and it is assigned to a URL named fileURL

(URLSession setup code precedes this)
...
...
var lines = [String]()
if let inputFile = try? String(contentsOf: fileURL) {
    lines = inputFile.components(separatedBy: "\n")
}

So now you have an array of strings representing each line in the file.

The bigger question is the format the source file is in. A CSV can be comma separated or it can be tab separated so the way you parse it into columns differs since you have to break the columns up based on the separator type. Let’s assume that it is comma separated.

The task is then to loop through each line and separate the columns into your data structure that would correspond to the CSV columns from left to right

You could do something like this:

for line in lines {
    let columns = line.components(separatedBy: ",")
    for column in columns {
         // your code to convert each column (which is a string) into the relevant data type according to your data structure
    }
}

Does that give you a bit of an idea of what you need to do?

How about trying to work with json instead? It will be easier in the long run and you can use dictionaries or areay to search afterwards… Check out our simple swiftyjson tutorial SwiftyJSON: How To Parse JSON with Swift

Okay, I understand most of that. Thank you @Chris_Parker! My question is now, how do I create a URLSession setup? I’ve looked through many YouTube videos and articles but they’re always pretty complex with suffixes that I’ve never seen. Also, just to clarify, the “fileURL” in (contentsOf: fileURL) is the web address of the file, correct? I tried putting in the URL but received the error, cannot convert type string to expected argument type URL. Sorry about the trouble. Thank you again!

Interesting! Thank you, I looked into this as well. Since I’m pulling data from an external source, wouldn’t I need to convert the csv to a json first? If I wanted the app to “self-update”, wouldn’t that require extra steps or is there code to firstly automatically convert a csv to json? :slight_smile: Thanks!

usually you need to set an some sort of API for that, i suggest looking into out notes app maybe it can help Full Stack iOS Notes App with Custom Backend - YouTube

For what it is worth I have figured out a way of downloading a CSV file from a remote location using a URLSession.

The CSV file layout is like this (the assumption being that when the CSV file is created from the spreadsheet, headers are suppressed):

Ford,Transit,White,2006,JG23XRDKJH000012
Holden,Kingswood,Blue,1968,HT68000000001234

For the purposes of the exercise I created a simple data model as follows:

struct Vehicle: Identifiable, Decodable {
    var id: UUID?
    var make: String
    var model: String
    var color: String
    var year: Int
    var vin: String


    init(id: UUID?, make: String, model: String, color: String, year: Int, vin: String) {
        self.id = id
        self.make = make
        self.model = model
        self.color = color
        self.year = year
        self.vin = vin
    }
}

The struct conforms to Identifiable simply to make it easier when viewed in a SwiftUI View. It also conforms to Decodable as I created a JSON version of the data to make sure I was able to retrieve that. There is an online tool to convert csv to json here: https://csvjson.com

The URLSession code to download the CSV file, is as follows:

    func getCSV(urlString: String) {
        var returnedData: [Vehicle] = []

        let url = URL(string: urlString)

        let task = URLSession.shared.downloadTask(with: url!) { (localURL, urlResponse, error) in
            if let localURL = localURL {
                if let urlContents = try? String(contentsOf: localURL)  {
//                    print(contents)
                    let records = urlContents.components(separatedBy: "\n")
                    for record in records {
                        let columns = record.components(separatedBy: ",")
                        let newEntry = Vehicle(id: UUID(),
                                               make: columns[0] as String,
                                               model: columns[1] as String,
                                               color: columns[2] as String,
                                               year: Int(columns[3])! as Int,
                                               vin: columns[4] as String)
                        returnedData.append(newEntry)
//                        print(newEntry)
                    }
                    DispatchQueue.main.async {
                        self.vehicles = returnedData
                    }
                }
            }
        }
        task.resume()
    }

Essentially the CSV file is a text file and in this case it is a comma separated file, so the process is to use the URLSession downloadTask which downloads and saves it to a temporary file which is accessible through the parameter labeled localURL.

Next step is to check that there is something returned hence the line
if let localURL = LocalURL {
and then extract the contents of that url into a single rather long string. Thanks to Paul Hudson for this tip: How to download files with URLSession and downloadTask() - free Swift 5.1 example code and tips

Next is to separate that long string into an array of records (effectively an array of Strings) with each record matching up with each Row in the table. It uses the newline character as the separator.

Next is to loop through all the records and for each record separate that into an array of columns using the comma separator as the means of delineating each column. Each element in columns matches up with each column in the original spreadsheet.

Next is to create a new instance of Vehicle and assign the columns to the properties and append that to the returnedData array.

When all the records have been processed, assign that to the viewModel array on the Main Thread.

@Chris Parker thank you so much! So I tried utilizing this in Xcode and everything seems to work well except for a few errors. “returnedData.append(newEntry)” under the declaration of newEntry returns the error of “Variable used within its own initial value”. Also, how do I print out each of these parsed csv lines into the console area? Lastly, would it be simpler to make all the data types of each csv header a string? (i.e. if “Date” was the column and 2282021 was the integer, why don’t I just name it as a string to keep it consistent?) Thank you so much again. :slight_smile:

@BananaNinja
Can you share your code that you have written to process the CSV file and your data model so that I can see what your naming convention is?

(edit) and a sample of your CSV file.

Yes. I hope I did the format right. It should be below. My csv file is here: https://firms.modaps.eosdis.nasa.gov/data/active_fire/c6/csv/MODIS_C6_USA_contiguous_and_Hawaii_24h.csv and it should take you to the download. I appreciate this so much, thank you again! :slight_smile:

struct File: Identifiable, Decodable {
    var id: UUID?
    var latitude: Double
    var longitude: Double
    var brightness: Double
    var scan: Double
    var track: Double
    var acq_date: String
    var acq_time: Int
    var satellite: String
    var confidence: Int
    var version: String
    var bright_t31: Double
    var frp: Double
    var daynight: String
    
    init(id: UUID?, latitude: Double, longitude: Double, brightness: Double, scan: Double, track: Double, acq_date: String, acq_time: Int, satellite: String, confidence: Int, version: String, bright_t31: Double, frp: Double, daynight: String)
{
    self.id = id
    self.latitude = latitude
    self.longitude = longitude
    self.brightness = brightness
    self.scan = scan
    self.track = track
    self.acq_date = acq_date
    self.acq_time = acq_time
    self.satellite = satellite
    self.confidence = confidence
    self.version = version
    self.bright_t31 = bright_t31
    self.frp = frp
    self.daynight = daynight
    }
}

func getCSV(urlString: String) {
        var returnedData: [File] = []

        let url = URL(string: urlString)

        let task = URLSession.shared.downloadTask(with: url!) { (localURL, urlResponse, error) in
            if let localURL = localURL {
                if let urlContents = try? String(contentsOf: localURL)  {
//                    print(contents)
                    let records = urlContents.components(separatedBy: "\n")
                    for record in records {
                        let columns = record.components(separatedBy: ",")
                        let newEntry = File(id: UUID(),
                                               latitude: columns[0] as Double,
                                               longitude: columns[1] as Double,
                                               brightness: columns[2] as Double,
                                               scan: (columns[3]) as Double,
                                               track: columns[4] as Double,
                                                acq_date: columns[5] as String,
                                                acq_time: columns[6] as Int,
                                                satellite: columns[7] as String,
                                                confidence: (columns[8]) as Int,
                                                version: columns[9] as String,
                                                bright_t31: columns[10] as Double,
                                                frp: columns[11] as Double,
                                                daynight: columns[13] as String,

                        
                        returnedData.append(newEntry)
//                        print(newEntry)
                    )}
                    DispatchQueue.main.async {
                        self.files = returnedData
                    }
                }
            }
        }
        task.resume()
    }

*there was an error for a hanging parentheses, so I added one above “DispatchQueue.main.async”.

(edit) The errors I got were “Expected ‘)’ in expression list” which was before I had added the parentheses stated above as well as “Variable used within its own initial value” for “returnedData.appened(newEntry)”. :grin:

OK, this is what the code should look like in order to work.

    func getCSV(urlString: String) {
        var returnedData: [File] = []

        let url = URL(string: urlString)

        let task = URLSession.shared.downloadTask(with: url!) { (localURL, urlResponse, error) in
            if let localURL = localURL {
                if let urlContents = try? String(contentsOf: localURL)  {
                    //  Split the file into an array of records
                    let allRecords = urlContents.components(separatedBy: "\n")
                    //  Remove CSV header line
                    let records = allRecords.dropFirst()
                    //  Process all the records
                    for record in records {
                        if record != "" {
                            //  Split each record into an array of columns
                            let columns = record.components(separatedBy: ",")
                            //  Create the array entry
                            let newEntry = File(id: UUID(),
                                                latitude: Double(columns[0])! as Double,
                                                longitude: Double(columns[1])! as Double,
                                                brightness: Double(columns[2])! as Double,
                                                scan: Double(columns[3])! as Double,
                                                track: Double(columns[4])! as Double,
                                                acq_date: columns[5] as String,
                                                acq_time: Int(columns[6])! as Int,
                                                satellite: columns[7] as String,
                                                confidence: Int(columns[8])! as Int,
                                                version: columns[9] as String,
                                                bright_t31: Double(columns[10])! as Double,
                                                frp: Double(columns[11])! as Double,
                                                daynight: columns[12] as String
                            )
                            returnedData.append(newEntry)

                            print("Data separated into Columns")
                            let dataString = "'" + columns.joined(separator: "', '") + "'"
                            print(dataString)
                        }
                    }

                    DispatchQueue.main.async {
                        self.files = returnedData
                    }

                }
            }
        }
        task.resume()
    }

The first line of the CSV file has to be removed as it is just a header line so that anyone importing the data into a spreadsheet has labels at the top of each column. We don’t need that so you will see I have added code to drop that record.

The other issue I discovered is that there is a blank record as the last entry so I took that into account so that if a record is not an empty string then process it.

1 Like

Oh! I see! I think this is going to be my last question since I think I understand the code except for one last thing. I keep getting Scope errors for the

where it cannot find “Type ‘File’, URL, or URLSession” in the Scope. I’m assuming I would plug in the url of the csv file I’m downloading which was referenced in the posts above in one of these spots and the error should go away? Or if not, where would I put the url string? Or something else to have Swift take the file in? Sorry, I guess you’d say I’m a newbie at Swift haha. Thank you so much, you’ve been so helpful. I really appreciate this again. :grin: :grin:

What is the structure of your overall project? Is it using the UIKit framework (storyboard) or SwiftUI framework?

I honestly have no idea. I’ve seen a little of storyboards but have not experimented with SwiftUI. I think Storyboards might be easier at the moment, so I’ll say UIKit. Thanks :slight_smile:

(edit) I’ve been using the code you’ve sent above in playgrounds to see if I could get it to print into the playground console.

I think you would be wise to take on some courses in App development so that you understand how to integrate data retrieval into your project and assign that retrieved data to an object (array) that you can then use as the source of data to display in the UI. The techniques involved are fundamental to you succeeding in this project.

1 Like

Yes. I have been learning from a book but got confused after I began this project. Do you have any resources that you’d suggest for this topic? Thanks!

You’ve already discovered Chris Ching (who owns Code With Chris and this forum) so you might consider taking on some of his courses to get you up to speed.

Start here: learn.codewithchris.com

1 Like

Okay, I will! Thank you so much for your help already. Just one last note, do I need to install Firebase or something from CocoaPods to help with importing data from external sources? I’ve looked into CSV parsers and things of the sort. Thank you. :grin:

No need for Firebase in this instance unless you want to store historical data related the data you are interested in. But even then, there are other options to Firebase.

1 Like

Okay, sounds good. Thanks! If you’re willing to return back to the original subject, I was just wondering, would I call the function as getCSV(urlString: “urlname.csv”) ? I tried calling it, but it wouldn’t call and I am not sure where to put in the original URL anymore. Also, what do “scope” errors mean? I feel like those are the only errors I’m encountering now. But if not, then that’s alright; I truly appreciate all of your help thus far!