Two of the most commonly used techniques when analyzing data are sorting, which rearranges the order of data (often alphabetically for text data, or from least to greatest for numerical data), and filtering, which removes data that doesn't follow a rule that is set. This video will cover the basics of performing both of these functions to a list in Dynamo.

- [Voiceover] Two of the most commonly used techniques when analyzing data are filtering and sorting. Filtering is a technique that removes data that doesn't follow a rule that we set, and sorting is a technique that rearranges the order of data. A lot of times, you'll sort text data in alphabetical order and numerical data from least to greatest or greatest to least. This video is going to cover the basics of performing both of those techniques to a list in Dynamo. Let's begin by applying the filtering technique. In this video, we'll be interested in filtering the list output from the RestOfItems node which is a nested list of an Excel file containing information about different types of retail shelves.

Let's begin with filtering. Filtering is a two-step process. First we need to identify the criteria or the rule that we wanna use to narrow down this list of data. Then we need to string together whatever nodes we need to test each item in our data against that rule. That output should be a list of boolean values which report true for items that meet the criteria we set and false for any items that don't. The second step is to take those results of the test and use a specific node that splits the data into two lists, one consisting only of data that produced a true outcome and the other list only of data that produced a false outcome.

For this example, let's filter our list down so that we're left only with items whose description equals accessories. Not shirts or pants, only accessories. Step one here is to set up our rule. In this case, the rule is that each list's index one needs to be equal to accessories. We can see that the list transpose node from a previous example was really helpful in producing exactly the list of data that we need here. Index one of this list is the description for each shelf type that came in from Excel.

Let's use the GetItemAtIndex node to extract just that one list of information. We can find the GetItemAtIndex node under core list in our node library. We'll also need a number input to specify the index. We'll set our number input to one to extract just the list at index one. Next we'll need to test whether each item in that list is equal to accessories. We can do that using the equals node which we can find in the operator section of the node library.

With this node, we'll be interested in testing each item in this list. We'll also need a string node for the Y input. Here, I'll type "accessories" and plug it into the Y input. Excellent, now we've run our test. Let's double-check whether it ran correctly now. We can see that items zero through five in our list are false.

If we compare that to the GetItemAtIndex list, we have shirts, shirts, pants, pants, footwear, footwear. Neither one of those equals accessories. So, false is the right answer. If we look at items six, seven, and eight, which are true under our equals node, they do, in fact, equal accessories. So, it appears that our test ran smoothly. Next, we need a node that's able to take this list of boolean values and use it to filter out our original list of data coming out of RestOfItems node. The node that we'll use here is called FilterByBooleanMask, which we can find under the core list section of the node library.

We can see that this node takes two inputs. It takes a list of data that we're going to filter, and it also takes an input called "mask." We know that the data that we're interested in filtering is the RestOfItems output. So let's go ahead and plug that into list. Mask is Dynamo's term for the list of boolean values that we just generated with our test. This naming might make a little more sense after we see how it performs. Let's plug both inputs in and see what comes out the other end. I'm going to drop in two watch nodes. We'll plug the in output to one and the out output to the other.

So now let's try to understand what happened inside that FilterByBooleanMask node. What Dynamo did here is that it looked at each item from the RestOfItems output, and it matched them up to each item in the equals or the boolean mask output. It matched them up by index, and anytime that a piece of data was matched to a true in the boolean mask, it was routed to the in output, whereas if we saw a false in the boolean mask, it was routed to the out output. Let's double-check that. We have true values at items six, seven, and eight of our boolean mask.

Let's head over to the RestOfItems list to see what items six, seven, and eight are. It seems that item six equals A1, seven is A2, and eight is A3, and in each case, item one of the sublist equals accessories. Let's compare those three to what came out of the in output of our filter node. Here we have A1, A2, and A3. Excellent, so it appears that our FilterByBooleanMask worked.

Now in the same way, any data that was matched up with false values was routed to the out output of the filter node. Here, we're seeing the rest of the data that doesn't equal accessories at index one. We're seeing pants, footwear, and shirts instead. So with just a couple of nodes, we've managed to narrow down our fullest of information to a smaller subset of data according to a rule or a criteria that's important to whatever analysis we might be performing. Because these nodes are all working together to filter data, let's select all of them and add them to a group.

I'll change the group title to filter by index one equals accessories. Now let's take a look at the process of sorting data which will rearrange the order of information. Now, there are a couple of different nodes in the library that can be used to sort data. The one we'll cover here is called SortByFunction, which tends to lend you the most control over the sort operation. We can find that node in the built-in section of the node library. That's a section we haven't yet visited during this course.

Again, let's use the RestOfItems output as a way of testing out our sorting feature. I'm going to move our new node group and our SortByFunction up a little bit to keep things tidy. You'll notice that the SortByFunction node takes two inputs. One of them is the list of data that we're interested in sorting, which will be RestOfItems. We'll go ahead and plug that in now. Next, we'll need to construct a function that Dynamo is going to use to sort this data. For this example, let's sort our list of data according to the width of each shelf type or the value that we can find at each sublist's index two.

So what Dynamo should is it should ask each item in our list, "What is your index two value?" and then, according to the answer to that question, it should reorder all of the information in this list. Now the process of constructing a function for sorting is very similar to the process of contructing a function for our ListMap node, and just like the ListMap node, there might be a little bit of confusion because it's going to feel like we're forcing data to flow backwards again. And again, we do need to leave another kind of confusing blank input somewhere in the function workflow. Anyway, let's go ahead and begin to construct our function.

We'll need to use a GetItemAtIndex node and a number input. Let's copy the ones that we use for our filter. I'm going to select them, copy them with control C, and I'll paste them with control V. I'm also going to change the number input to two since we're interested in looking at the width parameter which is at index two. Now remember, anytime we construct a function like this, we need to leave a single input within our function blank.

Keep in mind that the SortByFunction is going to take this cluster of function nodes and it's going to run every item of our list through them. The input that we leave blank here is how we tell Dynamo which input in our function node we're interested in running that data through. So in this case, we're interested in leaving the list input blank for our function. Now, we can go ahead and plug our two function nodes into the function input. Now, let's take a look at the output. Comparing the index to an each list, it seems that the list is now sorted from least to greatest.

Index two here is 18, then we see 24. Another 24, 36. Our list is reordered according to index two. Now, just to be clear, we didn't just rearrange the second index of each list item. We rearranged the order that each sublist appears in the overall parent list. Dynamo didn't drill down to the sublist level. Each sublist's data stayed intact. So there we go. In just three nodes, we've managed to sort or change the order of our parent list according to the criteria that we set.

Let's go ahead and group those nodes together too. Now that you've learned the basic sorting or filtering workflows, you're well on your way to analyzing data in Dynamo. Keep these techniques in mind anytime that you begin work on new data. It's common to start with a larger dataset than you actually need, and only a specific subset of the data will turn out to be valuable to you for the task at hand. Also, it's possible that sorting the data in a particular way can help you identify trends or spot mistakes, visualize information, or draw conclusions.

###### Updated

9/26/2017###### Released

3/29/2016- Placing and connecting Dynamo nodes
- Understanding Dynamo's data types
- Performing math functions
- Creating number lists and text strings
- Writing data to an Excel spreadsheet
- Creating points, curves, surfaces, and solids
- Analyzing geometry
- Linking a Dynamo-driven SAT into Revit
- Placing Revit families with Dynamo
- Creating Revit views and sheets with Dynamo

## Share this video

## Embed this video

Video: Filtering and sorting data