Utilities

extract_items and map_extract_items

A task scriptor is often used for is the extraction of data (most commonly into a table). extract_items and map_extract_items are aimed to ease the extraction of datapoints. They don’t cover every use-case, but the most common ones, and thus spare the developers of new scripts most of the repetitive work. Since examples working with the database would need to be highly specific to the customers data, we’ll use fictional demo-data.

This example shows how to extract the first names of a user and the first names of all of their friends. The demo-data contains two users, which are iterated over inside map_extract_items. The mapping defines, what data will be extracted from the given input-data and which name it will get in the result. The extraction of the first name of the user is simple: As you would select the first_name from a dictionary with dictionary["first_name"], you have to add "first_name" to the extractor-list. Similarly, you’d select the friends of a user with dictionary["friends"], which would be a list you’d need to iterate over. To signal to the extractor, that you want to iterate it, you can write three dots (...) called Ellipsis in Python. The following selectors then get applied to each element of the list. The result of this is then printed. Afterwards a table-file is made from the data, which is then converted to a list of lists representing the tables data, which is then shown in a Dialog.table.

The join_with-parameter defines how list-elements are concatenated. Setting it implies that all results are converted to string. Try removing it from the example code. You’ll see, that the friends names aren’t in a single string anymore, but in a list. This will also lead to the Dialog.table raising an error. This is useful if you want to modify the data before saving or displaying it.

#### scriptor ####
from viur.scriptor import *

async def main():
    demo_data = [
        {
            "first_name": "Peter",
            "last_name": "Jones",
            "friends": [
                {"first_name": "Darrell", "last_name": "Bryant"},
                {"first_name": "Julie", "last_name": "Brooks"},
                {"first_name": "Erin", "last_name": "Jimenez"}
            ]
        },
        {
            "first_name": "Amy",
            "last_name": "Stephenson",
            "friends": [
                {"first_name": "Christina", "last_name": "Allen"},
                {"first_name": "Joel", "last_name": "Alvarez"},
                {"first_name": "Melanie", "last_name": "Rivera"},
                {"first_name": "Erica", "last_name": "Nicholson"},
                {"first_name": "Roy", "last_name": "Weaver"}
            ]
        }
    ]
    mapping = {
        "First Name": ["first_name"],
        "Friends first Names": ["friends", ..., "first_name"]
    }
    extracted_data = map_extract_items(demo_data, mapping, join_with=", ")
    table_file = File.from_table(extracted_data)
    list_table_header, *list_table_data = table_file.as_list_table()
    await Dialog.table(list_table_header, list_table_data)

Note that map_extract_items recognises if you pass a list or a dict. If you pass a list, it will automatically assume the mapping applies to the items of the list and iterate over it.

That means that extracted_data = map_extract_items(demo_data, mapping, join_with=", ") does the same as

extracted_data = []
for user in demo_data:
    extracted_data.append(map_extract_items(user, mapping, join_with=", "))

If you want to extract more complex data, or want to transform the data, you need to iterate over the result of map_extract_items. In this example, we’ll do the same as before, but also extract the last names of the friends. Note that we removed the join_with-parameter, because we need the names as a list.

#### scriptor ####
from viur.scriptor import *

async def main():
    demo_data = [
        {
            "first_name": "Peter",
            "last_name": "Jones",
            "friends": [
                {"first_name": "Darrell", "last_name": "Bryant"},
                {"first_name": "Julie", "last_name": "Brooks"},
                {"first_name": "Erin", "last_name": "Jimenez"}
            ]
        },
        {
            "first_name": "Amy",
            "last_name": "Stephenson",
            "friends": [
                {"first_name": "Christina", "last_name": "Allen"},
                {"first_name": "Joel", "last_name": "Alvarez"},
                {"first_name": "Melanie", "last_name": "Rivera"},
                {"first_name": "Erica", "last_name": "Nicholson"},
                {"first_name": "Roy", "last_name": "Weaver"}
            ]
        }
    ]
    mapping = {
        "First Name": ["first_name"],
        "friends_first_names": ["friends", ..., "first_name"],
        "friends_last_names": ["friends", ..., "last_name"]
    }
    extracted_data = []
    for row in map_extract_items(demo_data, mapping):
        row["Friends Names"] = ", ".join(f"""{first_name} {last_name}""" for
                                         first_name, last_name in
                                         zip(row["friends_first_names"],
                                             row["friends_last_names"])
                                        )
        del row["friends_first_names"]
        del row["friends_last_names"]
        extracted_data.append(row)
    table_file = File.from_table(extracted_data)
    list_table_header, *list_table_data = table_file.as_list_table()
    await Dialog.table(list_table_header, list_table_data)