absorbent and yellow and porous is he

It turned over and over in my mind’s eye. I doubt it’s impossible. But I had just read a bunch of documentation. Squints. I have a Mendela Effect memory of an ArcGIS Notebook having a kernel switch button for R.

While there isn’t an out-and-out statement that, “you can’t use R inside of an ArcGIS Notebook in ArcGIS Pro”, everything is described like: - do your analysis in RStudio, then bring your results into ArcGIS Pro - arcpy can be accessed from R using the reticulate - R-based GP script tools are defined in a standalone R script file I mean, it is called the R-ArcGIS Bridge. A bridge, bridges, something. A gap? Gulf? Chasm? It implies discrete destinations. Depending on where you have lived, where you were raised, your mental image of a bridge might be big. I’m from Texas. We have some big-er (not everything bigger in Texas, that just Texan propaganda) bridges. I was raised near an artifical reservoir flooded during the 1930s. I have many memories of the very long drive across the bridge. Looking at the level of the lake to determine how bad the drought is. Or how relieved we are that the lake is full.

But when I close my eyes. It’s a looney tunes style bridge. I watched a lot of cartoons. When I push past that visualization, I see a foot bridge. Spanning a stream you could hop over. What’s the point? So your feet don’t get wet, I guess. Me? I have lots of memories of jumping over little streams (sometimes unsuccessfully, muddy antisc ensue).

let’s get our feet wet

and let’s be honest. If this were something that wasn’t considered impossible, there’d be some kind of support article about it. Surely a small blurb like, you can use it in an ArcGIS Pro Notebook! We’re on the main stage, Jack has some clever intro like, R you ready kids!? The crowd chants enthusiastically back, “AYE AYE CAPTAIN!”

Incrementing Number of Features

At previous organizations, I’ve done some fancy things with AssetIDs. And by fancy, I mean dumb. And by dumb, I mean, over-engineered based on spatial positioning for a grid system that was subject to change. I really like the pattern our staff came up with, it bakes in a create date, who collected it, and then doesn’t artificially constrain the total number of assets– –just assumes a person doesn’t collect more than 999 assets in a day. How does one increment features though? Through featuresets of course! But before we get to featuresets, it’s helpful to thing about how we’d do this on the desktop.

Prefix

This is more manual than I would actually prefer. I would love to simply ask the feature class what its name is and dynamically enter it but there is not yet any functionality for that. Since we’re using this from the Field Maps Designer we, as the author, know what feature class we’re working with.

var fc_prefix = "DRS"

So we’ll enter this manually, that’s okay I guess

User Initials

Creating this variable is most definitely dependent on how your users are entered into your ArcGIS Online organization or Enterprise Portal. The assumption is that you have some derivation of: firstname.lastname

Standardization is great. And sometimes you have to take an opportunity to implement it, like an automation for an AssetID.

I digress. Okay, we’re looking for initials, if I were working in Python I’d just do some string slicing here. In Arcade slicing is for arrays, so for text/string we work with syntax like Left, Mid, or Right.

var first_initial = LEFT("firstname.lastname", 1)

In order to isolate the last name, we’ll use the Split function on a known identifier, which would be the full stop “.”

Split returns the text as an array, so we’ll select the second index position [1].

var split_names = SPLIT("firstname.lastname") // ["firstname", "lastname"]
var last_name = split_names[1] // "lastname"
var last_initial = LEFT(last_name, 1) // "l"

Arcade can be kind of wordy if you let it be. You can combine all of those steps but eventually readability becomes an issue.

var last_initial = Left(Split("firstname.lastname")[1],1)

While one-liners work, sometimes they are a pain to maintain. We can keep it as a single variable and add some more legible structure to it.

var last_initial = Left(
    Split("firstname.lastname")[1]
    ,1)

And yeah, we totally got ahead of ourselves. How do we even get the username programmatically? With a Portal function, GetUser. It’s super neat and you can use it to do permissions management. Anyway, it returns a dictionary with the following values:

Our goal is to be able to do some things offline, so we are dependent on the username attribute only. As dictionary values, these can be accessed with dot-notation. The portal object is the current layer we’re working with which we can use the $layer global variable. But it could be any kind of Portal object if you know the itemID or its name.

var user_name = GetUser($layer).username // "firstname.lastname"

Now that we have a method for getting the username we can put it all together like:

var user_name = GetUser($layer).username
var user_initials = Upper(
    Left(user_name, 1) +
    Left((Split(user_name, ".")[1]), 1)
)

I added an UPPER to keep the initials looking tidy

8-Digit Date

Thanks to some convenience functions this one is fairly simple. We can use Today to grab today’s date. That will return a date object, we want to format it, so we’ll head over to the text functions for that piece.

Text will take any value and convert it into a text value, it has an optional format parameter that we’ll use to specify how we want our date to be outputted; the eight-digit pattern looks like: “YMMDD”

var date_eight_digits = Text(Today(), "YMMDD")

This is definitely less confusing than in Python

Incrementing Number of Features

At previous organizations, I’ve done some fancy things with AssetIDs. And by fancy, I mean dumb. And by dumb, I mean, over-engineered based on spatial positioning for a grid system that was subject to change. I really like the pattern our staff came up with, it bakes in a create date, who collected it, and then doesn’t artificially constrain the total number of assets– –just assumes a person doesn’t collect more than 999 assets in a day. How does one increment features though? Through featuresets of course! But before we get to featuresets, it’s helpful to thing about how we’d do this on the desktop.

AssetID LIKE 'DRS_JC_20241227_%'

var sql_query = `AssetID LIKE '${prefix}_${user_initials}_${date_eight_digits}_%'`

var current_features = Filter($layer, sql_query)

var feature_count = Count(current_features) + 1

DRS_JC_20241223_1
DRS_JC_20241223_10
DRS_JC_20241223_100
...
DRS_JC_20241223_9
DRS_JC_20241223_99
DRS_JC_20241223_999

var feature_count = Text(Count(current_features) + 1, '000')

function Calculate_Asset_ID(fp_prefix, fp_user_initials, fp_date_eight_digits) {
    var sql_query = `AssetID LIKE '${fp_prefix}_${fp_user_initials}_${fp_date_eight_digits}_%'`
    var current_features = Filter($layer, sql_query)
    var feature_count = Text(Count(current_features) + 1, '000')
    return `${fp_prefix}_${fp_user_initials}_${fp_date_eight_digits}_${feature_count}`
}

Hidden Pieces of Logic

I think that this is the whole reason why I wanted to write about this anyway. When I originally wrote this Arcade snippet, it did not have a test to check if the Asset ID already existed. Which meant, any time any asset was updated, it got a new Asset ID with a new date and the user information of the person who edited it. What a mess. So what we needed to add was a final test to check if the Asset ID was currently empty:

• return a new Asset ID if it was empty
• return the existing Asset ID if it was not empty

The easiest way to craft a test like this is with When, it takes some logic and returns a corresponding value, and a default value if it fails all of the tests. Our test will be to access the current feature’s Asset ID attribute. We can do this with IsEmpty, the global variable $feature, and some dot-notation.

When(
    IsEmpty($feature.AssetID) == True
    , // the value if true
    , // the default value
)

So we can tuck this into a variable assignment like this:

var asset_id = When(
    IsEmpty($feature.AssetID) == True, Calculate_Asset_ID(fc_prefix, user_initials, date_eight_digits),
    $feature.AssetID
)

I like having the logic test and result on the same line

An added benefit here is that we’re only accessing our featureset and running the calculations if it is actually needed– –this can slow down your workflow if the underlying feature class is very large.

And that’s it. When we put all the pieces back together we’re left with a pretty readable Arcade snippet that bakes in our requirements for a unique, iterable Asset ID, that records the created date, person who captured the data, and that is usable within the Field Maps Designer application.

function Calculate_Asset_ID(fp_prefix, fp_user_initials, fp_date_eight_digits) {
    var sql_query = `AssetID LIKE '${fp_prefix}_${fp_user_initials}_${fp_date_eight_digits}_%'`
    var current_features = Filter($layer, sql_query)
    var feature_count = Text(Count(current_features) + 1, '000')
    return `${fp_prefix}_${fp_user_initials}_${fp_date_eight_digits}_${feature_count}`
}

var fc_prefix = "DRS"
var user_name = GetUser($layer).username
var user_initials = Upper(
    Left(user_name, 1) +
    Left((Split(user_name, ".")[1]), 1)
)
var date_eight_digits = Text(Today(), "YMMDD")

var asset_id = When(
    IsEmpty($feature.AssetID) == True, Calculate_Asset_ID(fc_prefix, user_initials, date_eight_digits),
    $feature.AssetID
)

return asset_id

Looking at it now, I kind of hate it.

Why don’t we just shove everything in the function? I mean, if we only need to calculate these things sometimes, why are we calculating those other things all the time?

function Calculate_Asset_ID() {
    var fc_prefix = "DRS"
    var user_name = GetUser($layer).username
    var user_initials = Upper(
        Left(user_name, 1) +
        Left((Split(user_name, ".")[1]), 1)
    )
    var date_eight_digits = Text(Today(), "YMMDD")
    var sql_query = `AssetID LIKE '${fc_prefix}_${user_initials}_${date_eight_digits}_%'`
    var current_features = Filter($layer, sql_query)
    var feature_count = Text(Count(current_features) + 1, '000')
    return `${fc_prefix}_${user_initials}_${date_eight_digits}_${feature_count}`
}

var asset_id = When(
    IsEmpty($feature.AssetID) == True, Calculate_Asset_ID(),
    $feature.AssetID
)

return asset_id

That’s probably better

Since we’re doing everything inside of the function, we removed all the parameters. However, an argument could be made for some new parameters. The global variable $layer is doing a lot of work– –it’s acting as a Portal object for getting the username AND it is also returning the featureset of interest. Probably you’d want to separate those concerns.

I can’t unsee it. I put it in the gist.

Here’s the gist: ArcGIS Arcade - Construct a unique ID like: DRS_JC_20241227_001 https://gist.github.com/FeralCatColonist/a6d4f5a299dd9a5443687ed998fbb73f

Two Levels

This is where I had a moment of clarity. What I really wanted.

And it wasn’t 6 files (3 .bat files and their 3 corresponding .properties files).

It was a single file that I could schedule and then never worry about disaster recovery again… …you know, except if disaster were to strike… …and obviously a bi-annual disaster recovery exercise because you don’t want your first time testing your disaster recovery option to be when you actually have to use it to continue business operations.

I needed something like, a command line interface. Where I could state an option within the scheduled task and call it a day.

It ended up looking like this:

#on a weekly task
WebGISDR-cli-wrapper.py -full 4

#on a daily task
WebGISDR-cli-wrapper.py -incremental 4

With these extra options, I set the type of backup and the number of copies to retain as well.

And if you’re like me, the question you’re probably asking is, but how? And fortunately, that’s pretty easy to answer. I wrote a CLI wrapper in Python in about a hundred lines that really boils down to 3 functions:

1. prune_copies() - take a directory, search for the .webgissite files that match the selected backup mode, and keep only the number that you want to have on hand at a given time. This results in dropping a file every time before you run the WebGIS DR tool again.
2. set_WebGISDR() - open the webgis.properties file, change the parameter BACKUP_RESTORE_MODE parameter to match the user input
3. run_WebGISDR() - runs the original WebGIS DR CLI like it would normally run: webgisdr --export --file webgisdr.properties

We’ll go ahead and look at it in a logical sequence:

This is the flow of the CLI. We take some arguments, feed those to our main function, and then distribute those to the three primary functions as needed.

def main(backup_arguments: argparse.Namespace):
    print(
        f"\tbackup_mode is: {backup_arguments.mode}\n\tcopies to retain: {backup_arguments.copies}\n"
    )
    prune_copies(WebGISDR_backups_path, backup_arguments)
    set_WebGISDR(WebGISDR_dir, WebGISDR_filename, backup_arguments.mode)
    run_WebGISDR(WebGISDR_dir, WebGISDR_filename)

if __name__ == "__main__":
    arguments = configure()
    main(arguments)

For us to ask our user to input options at runtime, we have to do some prep-work in this configure function. This was my first time using the argparse library and it was fairly painless. Looking forward to doing more with it if I need to set runtime arguments again. There’s really not a reason to ask for more from the user other than what kind of backup or the number of copies that they want to retain. There are definitely more complicated backup copy management strategies but for 80% of use-cases this is likely satisfactory.

def configure():
    """Configures the user input arguments

    Returns:
        argparse.Namespace
    """
    parser = argparse.ArgumentParser()
    parser.add_argument(
        "mode",
        choices=["FULL", "INCREMENTAL", "BACKUP"],
        type=str.upper,
        help="""the type of WebGISDR backup to be performed: backup, full, or
        incremental. this input is case insensitive.
        """,
    )
    parser.add_argument(
        "copies",
        choices=range(1, 8),
        type=int,
        help="""
        the number of copies to retain. this is the number of backups
        only for the backup mode input/selected. this is exclusive of the primary backup
        which will run at the end of the script.
        """,
    )
    print(f"{spacer}\n{spacer}\nRunning WebGISDR Automation")
    return parser.parse_args()

And for a bit of context, here’s what that would look like if you ran the –help flag.

the WebGISDR Automation CLI Wrapper’s help menu

Running WebGISDR Automation
usage: WebGISDR-cli-wrapper.py [-h] {FULL,INCREMENTAL,BACKUP} {1,2,3,4,5,6,7}

positional arguments:
  {FULL,INCREMENTAL,BACKUP}
                        the type of WebGISDR backup to be performed: backup, full, or incremental. this input is case insensitive.
  {1,2,3,4,5,6,7}       the number of copies to retain. this is the number of backups only for the backup mode input/selected. this is exclusive of the        
                        primary backup which will run at the end of the script.

options:
  -h, --help            show this help message and exit

Moving on, we can look at the imports and constant variables. For most use-cases, the only thing that should need to be filled out is the WebGISDR_backups_path variable. Which, should live in a machine that isn’t the Portal component. Because. Disaster.

You’ll also notice that the import statements are fairly light. This is designed to utilize components of the standard library in Python so that a minimal environment can be set up on the Portal machine. Also pathlib is just an elegant library. When I first learned Python all the file management stuff was handled using the os library, I’ve found that pathlib much easier to handle and it is more readable as well.

import statements and constant variables

import argparse
import os
import subprocess
from pathlib import Path

# Variables
WebGISDR_filename = "webgisdr"
WebGISDR_dir = Path(r"C:\Program Files\ArcGIS\Portal\tools\webgisdr")
WebGISDR_backups_path = Path(r"\\some-filepath-here")
spacer = "-" * 50

Finally we’re at the primary three functions. Here’s prune_copies().

• We create a list of paths from the directory indicated by the constant WebGISDR_backups_path, where the backup matches our user input (using dot notation from our argument names that we defined in the configure function). All backup copies look like: yyyymmdd-hhmmss-TYPE.webgissite with this in mind, we can run a wildcard (*) string search to grab anything that matches our TYPE
• Sort the list in descending order, this means that oldest copies are at the end
• Create another list through a list comprehension so that we can have a pretty output that only includes file names
• Create a list of backups to be retained by slicing the sorted list according to our user input
• Create a list of backups to be pruned by slicing from the other direction
• Iterate through our prune list and use pathlib’s .unlink() to remove those from the file system

def prune_copies(webgisdr_backups_path: Path, backup_arguments: argparse.Namespace):
    """Prunes a list of files according to the number of copies.

    Args:
        webgisdr_backups_path (Path): a directory path to the .webgissite backups
        backup_arguments (argparse.Namespace): the user input arguments for backup and copies
    """
    print(f"Pruning backups")
    WebGISDR_backups = list(
        webgisdr_backups_path.glob(f"*{backup_arguments.mode}.webgissite")
    )
    WebGISDR_backups.sort(reverse=True)
    file_names = [file.name for file in WebGISDR_backups]
    backups_retained = file_names[: backup_arguments.copies]
    backups_pruned = file_names[backup_arguments.copies :]
    for file in backups_pruned:
        Path(webgisdr_backups_path, file).unlink()
    print(f"\tretained:\t{backups_retained}\n\tpruned:\t\t{backups_pruned}\n")
    return

Next we’re setting the parameter of the webgisdr.properties file to match the user input type in the appropriately named set_WebGISDR() function.

• We create a Path to the file
• Read in the file as a string
• Create a list of strings where each item is a new line
• Iterate through the lines and find the line with the parameter BACKUP_RESTORE_MODE. Replace this list item with a string that utilizes our user input
• Write the list of strings as string using a new line joiner

def set_WebGISDR(webgisdr_dir: Path, webgisdr_filename: str, backup_mode: str):
    """Sets the WebGISDR configuration file's BACKUP_RESTORE_MODE property

    Args:
        webgisdr_dir (Path): the webgisdr tools directory path on the arcgisportal machine
        webgisdr_filename (str): the name of your webgisdr configuration file, it is normally 'webgisdr'
        backup_mode (str): the backup mode that was input as an argument
    """
    print(f"Setting {webgisdr_filename}.properties: BACKUP_RESTORE_MODE")
    WebGISDR_properties = Path(webgisdr_dir, f"{webgisdr_filename}.properties")
    file_content = WebGISDR_properties.read_text()
    lines = file_content.splitlines()
    for i, line in enumerate(lines):
        if "BACKUP_RESTORE_MODE =" in line:
            print(f"\tcurrent value:\t{line.split(' = ')[-1]}")
            lines[i] = f"BACKUP_RESTORE_MODE = {backup_mode}"
    WebGISDR_properties.write_text("\n".join(lines

Lastly, we just need to run the standard WebGIS DR tool in the function run_WebGISDR().

• Change directory to the WebGISDR tool within the Portal installation
• Craft a string that will be used in the subprocess.run() function, this is analagous to opening a command prompt terminal
• Run the webgisdr command

Python code describing the run_WebGISDR() function:

def run_WebGISDR(webgisdr_dir: Path, webgisdr_filename: str):
    """Runs the WebGISDR command-line utility

    Args:
        webgisdr_dir (Path): the webgisdr tools directory path on the arcgisportal machine
        webgisdr_filename (str): the name of your webgisdr configuration file, it is normally 'webgisdr'
    """
    print(
        f"Changing Directory to {webgisdr_dir}\n\trunning standard WebGISDR utility now\n\n{spacer}\n{spacer}"
    )
    os.chdir(webgisdr_dir)
    cmd_webgisdr = f"webgisdr --export --file {webgisdr_filename}.properties"
    subprocess.run(cmd_webgisdr, shell=True)
    print(f"{spacer}\n{spacer}\nAll Operations Complete\n{spacer}\n{spacer}")
    return

And that’s it. A fairly painless automation for setting and forgetting an out-of-the-box disaster recovery solution. When it runs it looks like this:

If you’d like to use it in your own deployment you can find it on my GitHub here: https://github.com/FeralCatColonist/Spatial-Grimoire/blob/main/WebGISDR-cli-wrapper.py

ArcPy Minor Upgrades are a Minor Headache.

Two years ago I wrote a long article on how to create a standalone ArcPy environment; normally, I’d say it is worth a read but its probably out of date. I’ve seen a few more articles since then covering the same subject matter so I’m hoping that this a more common practice. Why am I writing this? Yesterday, Esri released a new minor update to ArcGIS Pro from 3.1.x to 3.2.0 and judging from the What’s New page this is one of the more feature-rich releases to the software since 2.7.x.

https://pro.arcgis.com/en/pro-app/latest/get-started/whats-new-in-arcgis-pro.htm

With a new minor release comes new binaries, meaning that any Conda environments created within the ArcGIS Pro GUI or ArcPy installations created in a standalone fashion need to be freshly installed. You’ll know if this happens when you see an error like this:

Normally one could just run a command like:

conda update some-library

Because of the binary situation above, this does not work with the ArcPy library. We have two options for removing and re-installing ArcPy which I’ll cover. Esri has a convenience feature for “upgrading” your ArcPy environment; however, the actual mechanism works like this:

If you are using ArcPy in a standalone kernel for Python, and you have, or are planning to, install the new minor release update (3.2) to ArcGIS Pro you will need to reinstall ArcPy in your Conda environment. Let’s look at a few options that you have for performing the reinstall; we’ll imagine a few different scenarios.

Scenario 0 - You Can do a Standalone Installation of ArcPy!?

Yes. It’s super easy and this is how I do it! From an Anaconda prompt window, we’ll do the following using the placeholder ENV_NAME for the name of your standalone Conda environment. Feel free to name it whatever you like.

conda create -n ENV_NAME
conda activate ENV_NAME
conda install -c esri arcpy

The first line creates the environment, you’ll need to press the “y” key and then the “enter” key to complete the installation. The second line will activate this new environment. The third line installs ArcPy and all of its related packages into it, this also includes the ArcGIS API for Python.

https://anaconda.org/esri/arcpy

Scenario 1 - A Simple Scenario

I just installed ArcPy and nothing else. Congratulations! You can remove and re-install ArcPy without thinking about it too hard.

conda deactivate ENV_NAME
conda remove -n ENV_NAME --all
conda install -c esri arcpy

The first line deactivates your environment if it is open already. The second line removes all packages. The last line will reinstall ArcPy. That’s it, your environment will continue to work as expected and won’t break any of your scheduled tasks if you had them written against the python.exe path within your kernel.

Scenario 2 - I, uh, Installed a Few Other Packages

This doesn’t surprise me. Anaconda is a package manager. You use it to manage packages. You like to dabble in GeoPandas and other libraries of substance. Me too. If you know exactly what you’re looking for, you can trawl through a .yaml file:

conda activate ENV_NAME
cd "some path where you want the exported file"
conda env export > some_file_name.yml

The ArcPy library has a lot of dependencies so browsing it like this isn’t really my cup of tea. Although, a .yaml is a great way to share your environment with someone else. I think an easier way to look at your package installations is to look at your revision history. This is Anaconda’s version control, when you access it, the output is a numbered list of the libraries and their versions installed at each command:

conda activate ENV_NAME
conda list --revisions

This batched version of downloads should help you to figure out which libraries you downloaded (and in what order) so you have a better idea of how you need to re-create your environment. ArcPy is normally the first install I make after creating the environment, so I rolled back my environment and then re-installed ArcPy like this:

conda install --revision 0
conda install -c esri arcpy

Your syntax will likely be different depending on the number if libraries you need to include and in what order of installation those happened.

And it’s kind of that easy. Admittedly, not as easy as upgrading a library with the update command but such is the life of a licensed software library.

I Don’t Know the Difference Between CONTAINS and WITHIN

Fortunately, that’s not true anymore. However, I did go through a support ticket process with #esri to finally come to that realization. To be fair, the differences are laid out in the #arcgisarcade documentation (a bit better in CONTAINS than WITHIN). The short version is like this:

• CONTAINS, is there something inside of the feature (a container)?

• WITHIN, is the feature inside of something (a container)?

A good idiom here would be, two sides of the same coin. Similar operations that differ in their directionality.

CONTAINS() documentation:

WITHIN() documentation:

A simple tweak to the diagrams provided could make them more explicit:

And if you’ve stuck with me this far, here’s the pay-off; a second revision showing how that original code should have been written, with variable names that can be linked back to the documentation:

//https://developers.arcgis.com/arcade/function-reference/geometry_functions/#contains2
var insideGeometry = FEATURESETBYNAME($datastore, "BuildingFootprints")
var containerGeometry = $feature
var features_within_container = CONTAINS(containerGeometry, insideGeometry)

var calculated_area = DefaultValue(Sum(features_within_container, "Shape_Area"), 0)

CONSOLE(calculated_area)
return calculated_area

The Loneliest Number

The term is meant to quantify the number of project members that would need to be absent to create chaos. A factor of 1 would be the riskiest model, with each increase providing substantially more protection to your organization. However, with most GIS shops being a one-person operation, it may seem that your Bus Factor could never be greater than 1. At this basic level, there will almost always be a disruption to the flow of work. Despite a quick turnaround in hiring, the interruptions could span months. It doesn’t take much to imagine the horrors of crawling through an unknown network, digging into the husk of a former employee’s Outlook data file for project archives, or being emailed a pdf map to replicate and then searching through mounds of “Export_Output”-style shapefiles and map templates with names like “final2_maprequest_new”.

At this size of organization, one of the best outcomes that we can hope for is mitigation through documentation. While not the most exciting of topics, project documentation is vital and plenty has been written about it by people far more knowledgeable than me. I’d like to hit on some of the highlights and ethos of where to spend your energy in the realm of documentation.

For digital work, data is both nonrival and has near zero marginal cost to replicate; meaning, the largest cost to an organization is the development of the deliverable, afterwards it can be used and maintained with minimal upkeep. As such, keeping track of template files is incredibly important– –this is relatively easy with an MXD or APRX but requires additional effort with documenting commercial off-the-shelf (COTS) products that may have a large degree of customization (I’m looking at you ArcGIS Online). With template files, one of the greatest dangers is not updating source destinations after a major infrastructure upgrade; think about the last time your GIS database had a name change or when you migrated networks?

The other foundational rule of documentation is one of the most overlooked areas of data governance generally, metadata. Your data needs a dictionary. It needs an intent. And if you aren’t aware of the data provenance then how will it be authenticated, updated, or re-created? If you would like a slightly terrifying exercise, try running this meta-data dump

gist.github.com/FeralCatColonist/7504ab0c380542f12220fe62142082c1

on your GIS repository and emailing it to your supervisor. Again, the goal of a one-person shop is not to prevent disruption but to ask yourself, if a competent replacement was hired, would they be able to step into this role with minimal effort?

A Broader View of Indispensable

Who on your team represents a single point of failure or bottleneck? As a thought experiment, go through each member of your team and enumerate their projects and skillset. If you’re having trouble with either of those, or their skillset is unique, you’ve found a Bus Factor of 1. This can be repeated for known deliverables or other statutory responsibilities. The management types among us practicing a “need-to-know” style represent a Bus Factor of 1 as well. While Saturday morning cartoons taught me that knowing is half the battle, the more exciting part of this story is what to do next.

Ideally, your Bus Factor should be as large as your team. If this strikes you as a hot take and you want to talk to me about the merits of specialization or titled roles, feel free; I’ll respond with this: you should be investing in your employees. Full stop. Aside from that, there are definite benefits for having a 360-degree view of your unit and how it interacts with the rest of your organization. This doesn’t mean that the efficacy of everyone’s skillset needs to be the same, but everyone should be familiar, and able to interact, with a workflow if the need arises.

There are many ways to increase the Bus Factor for common deliverables: lunch and learns, side-by-sides or shadowing, and how-to videos. The lunch and learn serves multiple purposes, it works as a team building activity, practice for presentations, and familiarizes staff members with a workflow; in the Before Time, this could be a fairly fun event with catered food from a local restaurant. In the Pandemic Time, this becomes a remote affair with lunch brought to you by your pantry. In the New Normal, a side-by-side or shadowing becomes a little less cramped as individuals don’t need to physically huddle around a desk and everyone can have a front seat using a screen sharing technology. How-to videos are great and we’ll delve into that in the companion video! Spoiler alert, it’s not complicated and the tools to do it are free.

In terms of programming and knowledge work, increasing the Bus Factor takes a bit more effort or rather, a dedicated framework. My recommendation here is to adopt an agile methodology that makes use of the following artifacts: personal Kanban boards, code reviews, and version control. A publicly visible Kanban prioritizes what’s important and allows the collaborative and inquisitive team to cross-train on new projects that enter the workspace. While a common practice in software engineering, code review is unheard of in most spatial departments. For the uninitiated, code review is a public assessment of programming scripts in terms of style, patterns, and overall usability. It’s an excellent way to inform team members about the mechanics of an automation and to make sure that coding is consistent and legible amongst peers. This goes to the heart of the Bus Factor: your scripts are useless if the only person who knows how it works is you. Finally, version control attempts to cushion your code base against significant errors by providing provenance; when an item changed and why, and the ability to rollback your code to a point where it actually worked. You should be using Git. It’s easy to learn and there are a variety of platforms to choose from. Consider also opening your source code (through platforms like GitHub or GitLab). It’s a way to collaborate across organizations and to benefit beyond the numbers/talent of your immediate team.

An Ounce of Prevention

The axiom attributed to Benjamin Franklin places a 16:1 ratio on preventing a disaster as opposed to the cost of responding to it; in a world of mega-fires, pandemic, and Snowmageddons that ratio seems somewhat provincial. In terms of our own organizations, the disruption of any instantaneously available deliverable incurs an astronomical cost above what was effectively zero. As horrifying as these worst-case scenarios may be, people are pretty great at being complacent. From a biological perspective, it probably helps us to live longer if we’re not in a constant state of anxiety about the future. Our underlying motivations for increasing our Bus Factor do not need to be rooted in these sorts of negative emotions, and can instead be undertaken in a spirit of commitment to transparency and life-long learning.

One of the great tragedies of the digital revolution is the abuse of the principle of least privilege, restricting access to data and programs unless ordained as necessary for a certain user or workflow. Aside from over-securing data and inhibiting innovation, this process tends to bleed well beyond the system architecture and deprives an organization of its greatest resources: access to human capital. Transparency is both prevention and cure for this state of affairs. Obviously, building a culture of transparency is harder than maintaining one. Some ideas to increase transparency are daily standups, personal Kanban boards, weekly one-on-ones, and open-door policies for executive-level meetings. The undercurrent here is a level respect for employees at all levels of the organization and the idea that contributions can come from anywhere. These contributions don’t happen in a vacuum, work must be visible!

During this Second Machine Age, some of the defining hallmarks are velocity, volume, and variety of information. If there was ever a time when a person could satisfy themselves with never learning anything new, that time has most certainly passed. Aside from the benefits of personal actualization, our current environment demands that we constantly learn to maintain our position within it. At your organization learning should be encouraged, subsidized, and celebrated. The absolute worst outcome here is failing to invest in your employees; a self-inflicted anemia, starving them of new knowledge and skills. In the pandemic, we have been inundated with an enormous amount of remote learning opportunities that were previously restricted to paid, in-person offerings; a future struggle will be to maintain our access to these resources when the masks come off.

With the end in mind, both figuratively and literally, the Bus Factor is less klaxon and more health gauge for your organization. While some areas of your organization may be quite technically advanced, your overall maturity is low if those capabilities are based on personality and not institutionally defined. The paths to increase your Bus Factor follow many best practices of winning organizational cultures: personal growth, collaboration, and effective communication. In that sense, the Bus Factor is less about the destination and more the journey. Taking the time to document your work, working well with others, and learning new things, saves time in the long-term and fosters a more attractive workplace overall.

Companion Video Demonstrating Common Video Documentation Workflows https://www.youtube.com/watch?v=S-biFgYKdNY

In Closing

This latest release really is a watershed moment for creating stable business systems that are programmatically managed. Different environments can be deployed allowing a greater degree of customization in terms of feature enrichment, data cleansing, and the sundry extract-transform-load chores that define the modern enterprise. Honestly, I’m totally stoked. Too many times I’ve held off on major upgrades because of the labor involved with recreating my Python environment. Hopefully, this clears up any confusion for those uninitated with managing Anaconda environments. If you have any questions feel free to ask!, or check out Anaconda’s excellent documentation!

The Longest Wait

Finding my first job was a Sisyphean task. Tailoring my applications and cover letter for the hopes of getting an interview, interviewing, and then, radio silence. In those days of desperation, the idea of breaking into the industry seemed mythic. Eventually, I landed an opportunity digitizing cadastral records.

For the first time in my adult life, I had a semblance of stability. Naturally, I immediately mapped out a life plan. Eerily, it looked similar to the expectations for any employer’s fabled entry-level position:

• 3-5 years’ experience
• Web/Server knowledge
• Programming expertise in multiple languages

The issue, as I came to realize it albeit several years later, was that I had done very little to achieve any of these goals. Other than, of course, waiting. I went to work every day, left at the end of my shift, and spent the weekends laid about, as if I deserved nothing more than to rest after a week’s worth of mind-numbing work.

This realization came quite unexpectedly when the company for which I worked had several bouts of hiring. Fortunately, I had a boss who saw value in both delegating and the collaborative process. We were expected to sift through the hundreds of applications submitted, rank applicants based on our own personal subjective scale, and then reconvene with a final vote as to who should get invited to interview in-person. This consensus hiring continued into the interviews, which were an entirely different creature. Often, those that had the most experience on paper could not articulate core technical concepts outside of their daily wheelhouse. When asked about their motivation or future aspirations, the response was something habitual about “liking to make maps”, “enjoying the cartographic process”, or “being good with computers”. I suppose it is somewhat difficult to get excited about cadastral record-keeping; however, I imagined myself on the other end of the interview process and found my own answers to be equally lackluster.

I had been spoiled with a promotion a year since my initial hiring but this forward momentum began to falter after our manager left. A replacement was appointed from within our team without a competitive process– –which was not unusual considering the size of the company and the relatively clear lines of hierarchy. Team consensus was abandoned, new positions were opened and appointed. Suddenly it seemed that the metaphorical emergency brake had been applied full force. The months dragged on. I watched as my more ambitious peers moved on to other companies.

I felt bound by my own lack of imagination. I had spent years grinding away, gaining “experience”, but in reality, I had done very little to progress. I found myself at my own phantom stop. I had been pacified by the monotony of my own work, sitting at my desk imagining that each day brought me closer to advancement, the next job, a corner office with a placard on my desk. The harsh reality was that the bus I was waiting for was no longer running; that line had ceased regular service even before our profession was defined by a graphical user interface. The idea that somehow being next in line, or that time served was enough to propel a career, to a destination that I hadn’t even bothered to articulate, was an illusion born of inexperience and wishful thinking

Forward Progress

Identifying a problem is not a solution in itself, but it does allow you to formulate a remedy. I began interviewing. Admittedly, interviews were much easier to acquire with the requisite years of experience that I had gained to date. Having been on the driving end of several dozen interviews, the exchange seemed more mundane as I better-understood business requirements and their relation to spatial technologies. At that point in my career, it would be disingenuous of me to say that I understood accurately how to treat the malaise of idleness.

Frankly, it was not until my current position that I began to adequately conceptualize habits that contribute to forward progress. As much as I appreciate a top ten list, for me, there are only two salient actions: getting your foot in the door and being able to make an impact once you’re there; or in other terms, pairing networking with research and development. Previously, I had stumbled on elements of each but never had pursued the broader themes in any deliberate sort of way. As a consultant, my job was essentially to network. Learn their name, understand the situation from their perspective, solve problems, smile when you talk, be both pleasant and approachable. Our services were on retainer, no penalty contracts; we retained clients based on how we treated them.

The idea of attending professional events, presenting at conferences, writing articles, or setting up a regional Slack channel, is deeply unnerving to me. Almost mechanically, I force myself outside of my comfort zone; ask questions at the end of presentations, introduce myself to strangers, keep business cards on hand, and practice my 10 second response to “what do you do in your position?”. The result is somewhat comical as I battle to hear myself and others over a pounding heart, rushing blood, and a near constant stream of internal dialogue. This a set of skills that I neglected to develop outside of a professional client-consultant relationship; however, it is something that can be taught and learned. Maintain eye contact, don’t overpower a group conversation, and be comfortable with listening.

Rote actions will appear as hollow as they feel when they lack appropriate intent. A hammer for the sake of the object itself is only a showpiece; the utility materializes in the presence of a nail. When meeting new people, or reconnecting with acquaintances, my goal is always to learn. Every individual has expertise gained through virtue of their position, their work environment, and their life experiences. People are generally interested in discussing things that they know; in most cases, people will discuss themselves. This dynamic works well with patience, practice, a few leading questions, and an easy pivot. “What are you hoping to implement in the near future”, “what’s been the most challenging project”, “how did you implement that”, and “why is that”. Cultivating a genuine desire to explore and discover the stories of others has catalyzed my new conception of networking.

Additional positions were predicated on some of the unique experiences that I had acquired as part of the official workday at my first job. Client site visits for sales and installations, web service administration and application building, and some light automation. It was beyond fortuitous that a first position would span such a wide vertical integration of skills; the second position was far narrower. The pay was better but contractual, the company was national, and suddenly I felt expendable. The notion of a “safe” job had evaporated, along with it, my complacency. I began spending all my free time developing the traits that made me marketable.

Destination Ahead

My research and development (R&D) time was exclusively during lunch hours. Although restricted to the technology that the organization currently used, I was more than free to use our software to test new workflows, run different analyses, and prepare articles and presentations. Outside of specific software applications, I organized my lunch hours around interesting webinars and explanatory YouTube videos covering a variety of topics. And I read. Constantly. The wide availability of both digital and print materials makes most subjects accessible to those that are willing to spend the time to seek them out.

Sacrificing personal time and pursuing R&D outside of working hours has several distinct advantages. Chief among them, working with other software and versions that are unapproved. In the Esri realm, your organization may not have certain extensions or be several years behind the current software release. An ArcGIS Home-Use license costs $100 per year and comes fully loaded with all extensions, allowing the ability to hone skills that you would not otherwise have access to, and remain on the cutting edge of functionality. Outside of Esri products, there is an entire world of spatial technology not commonly used in American workplaces. QGIS is free and delivers the most up-to-date spatial functionality, including working in 3D; it also natively uses modern file formats like the GeoPackage. Moving beyond graphical interfaces, any sort of programming is made easier with unfettered admin rights. Python and R deliver incredibly fast, capable, and visual solutions to geospatial problems; paired with Project Jupyter, programming is becoming more collaborative and approachable to a wider audience.

As providence would have it, my current position quickly followed my second. Here, my interests are closely aligned with those of the organization: use spatial technologies to enable efficiencies and analysis-driven decisions. I had spent years effectively sitting on a bench, waiting for something to come and deliver me to a future that was nebulous and undefined. When I finally woke from my self-induced daydreams, I was startled into action, but without a clear destination in mind. I wasn’t sure where I wanted to go, but I had an idea of the skills that I would need to be successful.

My current organization is aggressively pursuing a transformation into a High Performance Organization (HPO). As such, opportunities abound to study organizational dynamics, performance, and agility. Among other things, this has impressed upon me the importance of a structured R&D program for employees. The idea that knowledge workers are only as effective as the knowledge that they possess, makes imperative the continuing acquisition of knowledge and skills. Today’s high-octane pace of technological change argues that the stagnant mind becomes increasingly outdated and obsolete. Hiring an employee without a training infrastructure is akin to purchasing a vehicle without a thought given to refueling it. The cost of doing business includes investing in the continuous education of your workforce. Acceptance of this new reality, like many things, is a spectrum. While few of us may be in a position to dictate organizational policies, we all have the agency to pursue personal growth and influence others through our example.

My journey continues. Within my team, our measures of success involve setting aside time for R&D; we use the term “Q2”, short-hand for Quadrant 2 from the Eisenhower Matrix and are expected to dedicate a minimum of 10% of our working hours towards this goal. In addition to this, I still set aside my lunches for reading and webinars. The more that I learn, the better idea that I have about what else I should know. And the wider that I branch out, I find myself less satisfied with my current level of expertise.

In life, destinations require planning and measurable effort. The bus isn’t real. There does not exist a vehicle, actual or metaphorical, that will spirit you away into a rosy and productive future. Our personal phantom stops may not be physical but their effects are tangible. Often times it is difficult to discern these mental decoys because they are so comfortable. We’re bored. Cruising on autopilot. Waiting passively for something to happen for us. At the heart of this illusion of security, a lack of skills development and refusing to learn from others. While these mental enclosures may be a figurative shelter of sorts, the world continues moving forward. And the complacent wait. And wait.

Unintentional Phantom Stops a Photo-Map Collection by Jordan Carmona, GISP

Here are a series of transit shelters throughout the city that currently sit unused. They are in different stages of disrepair, depending on the area of town and stewardship by adjacent property owners. On my commute, I pass the same shelter. I continue to be surprised by the feelings of loss, desolation, and longing that it evokes. That became the seed that grew into words. With the passing of conventional transit strategies, my hope is that these relics are repurposed into something more useful; I would hate to see them demolished as if this section in our history had never happened.

Cheval Blanc 1947

As part of the cohort from the analog generation, I was thrilled to experience the rapid uptick of technology in the late 20th century and the excitement that each year would bring massive improvements over the last. Looking back, perhaps I was too optimistic on how quickly or efficiently we would use the seemingly un-ending bounty of tech. The first blu-ray movie that I ever saw was Pixar’s Ratatouille. The perceived villain of the story, Anton Ego, is a feared critic whose harsh review caused a reversal in fortunes for Chef Gusteau and his self-titled restaurant, which neatly sets up story line and plot that again culminates with the expectation of another scathing review. In this story, we’re neither the anthropomorphic hero nor his human compatriots; just prospective diners, deciding where to spend the currency that is our time and attention.

As a pairing, initially, I thought Cartography. and The Location Advantage complemented each other well. Map making and actionable analysis. Data presentation in two forms, the artistic and the practical. Declarative and persuasive. Coming off the heels of the SCAUG annual conference, I felt energized enough to attempt these two courses in tandem. As defined by Esri, these courses:

• Cover a single theme

• Involve four to six weeks of instruction, with two-three hours per week of study

• Require registration but are free to take

• Run once or twice a year with firm start and end dates

• Provide certificates to students who complete the course material

• Introduce students to subject matter experts from across Esri and its community

• The Evolution of Esri’s MOOC Program, Part IV note from the future, resource links that are now dead have been removed

Signing up is probably the easiest part of the process, the interface is handled through the My Dashboard section of esri.com/training; depending on your organization’s internal policies, your named user may not be “enabled for Esri access.” If so, treat yourself to a free (Essentials Plan) ArcGIS Developer account (developers.arcgis.com); it is similar to the ArcGIS Public Account but has access to premium content, premium apps, training labs, and 50 free credits a month. After signing up, the MOOCs will appear as training items in your dashboard, clicking on one will bring you to that course’s home page, and from there you “enter” the course, another dashboard-type situation.

Chew it Slowly

Like any pairing, comparisons between two items experienced concurrently are inevitable. As much as I tried to avoid it, I had a definite preference going into the MOOCs. My favorite course in school was Applied Retail GIS; case studies, ArcGIS Business Analyst Online (BAO), and applied location analytics. I was dreading a course on cartography, probably one of the weakest aspects of my professional work. As part of our organizational ethos, we strive for self-improvement– –I was determined to improve myself in this aspect. Esri has opted for a more traditional experience with the majority of their MOOCs, content is incrementally unlocked on a weekly schedule and a syllabus has been prepared ahead of time. This meant that each week I experienced the pairing anew. A tot of Cartography. followed by a chaser of The Location Advantage.

You Must Try Things that may not Work

Over the next 6 weeks, the time-demands were pretty significant. If you have the ability, I would whole-heartedly recommend carving out research & development time during your working day. Doing more than treading water in the ever-expanding ocean that is spatial technology requires us to deliberately increase our skill set by trying out processes and discovering new ways to do things better. My R&D time was largely spoken for, which is why I opted to spend my lunch hour completing the MOOCs, even still, I ended up doing some of this work at home as well. In the end, what was left?

Understanding the Esri MOOCs requires understanding MOOCs in general. The biggest deficiency of these programs happens to be their best selling points, low barriers to entry and accessibility from anywhere form a disconnect between the student and the material. Unlike compulsory education, there is no school building that you go to daily, no teacher to reprimand you for failure to meet expectations, and no peer pressure to complete the learning journey together. While MOOCs enjoy high rates of registration, completion rates lag far behind. The Esri MOOCs are not exempt from this trend, using viewing rates from the Cartography. videos hosted on YouTube as a proxy, the approximate attendance rates for Week 1 started at 20,332 and finished Week 6 at 3,440. While the 83% decrease overall seems shocking, the largest numerical drop is between Week 1 and Week 2 (6,088); if viewed from Week 2 to Week 6 the attrition rate is a more realistic 44%.

Esri has opted for a more traditional experience with the majority of their MOOCs, content is incrementally unlocked on a weekly schedule and a syllabus has been prepared ahead of time. Dialogue is mostly uni-directional; although, there are some open forums during some weeks of the MOOCs (course instructors participated!), most of the time a student is offered a one-and-done response to finish the section each week. Each course did include an ask-me-anything session hosted on Esri’s GeoNet, the participation in each course varied dramatically with 18 comments in The Location Advantage and 165 comments for Cartography.

Hungry for More

I will admit that taking two courses concurrently was a bit challenging, and perhaps not something that I will repeat; but it’s hard to say no to a great pairing. In terms of time spent, I learned things that I did not know, that I had forgotten, and had never considered. The technical aspects were great in terms of learning more about functionality that I had not yet explored in ArcGIS Pro and Business Analyst Online, and from that view, these courses were a wild success. The ability to learn about a technical subject and to have fun while doing it is the sort of privilege that is rare amongst most professionals, and meeting face-to-screen with some of the giants in the industry is even better. I am looking forward to seeing Esri continue to dedicate resources to producing original content, keeping existing courses current, and hopefully extending some screen time to individuals outside of their immediate organization.

The climax of Ratatouille sees the food critic Anton Ego discover the true identity of the famous new Chef at Gusteau’s, and despite this, write a surprisingly unabashed endorsement of the restaurant. In our conclusion, the reviews are mixed. Many aspects were well-thought out, creative, and contribute to the continuing evolution of MOOC best practices. Other features seemed ill-prepared, unimaginative, and posed a general disregard for the appetite of the intended audience. I will return again, and my expectations will be high.

The people at Esri are enthusiastic about their work and that enthusiasm deserves to be shared as widely as possible; I’d like to thank them for stepping out and taking risks with distributing their work in this fashion. The next round of MOOCs will launch September 5 and run through October 16, on the menu is Cartography. and Do-It-Yourself Geo Apps. Another two courses will be available starting October 31 and ending December 11, Going Places with Spatial Analysis (Season 2) and Earth Imagery at Work.

The IKEA Effect Unpacked

The IKEA effect was based on the analysis of a series of experiments where certain participants constructed and bid on IKEA storage boxes or folded origami, while others bid solely.¹ These experiments measured value by comparing willingness-to-pay against the different classes of products assembled by participants and assembled by others, with the caveat that the assembly process must be fully completed.

These first two experiments focus on identifying the existence of higher valuation for self-assembled products and finding relative benchmarks for how much higher those valuations are in comparison to different standards. Experiment 1A shows that participants were willing to pay a 63% premium for a simple box that they had constructed themselves, as opposed to an otherwise identical box that was pre-constructed. This is the broadest sense of the IKEA effect and justification for the namesake. Perhaps even more interesting was the next experiment measuring magnitude.

First, participants, termed builders, were asked to create an origami frog or crane, and offered a chance to purchase their own creation. Second, a group of experts constructed the same types of origami creatures. Third, a different set of participants, termed others, were asked to bid on both sets of works– –setting a market price for papercraft animalia ranging the spectrum from amateurish to expert. The resultant average bids ranged from $0.05 - $0.27; builders valued their artistic endeavors at a premium of 23 cents, while others were willing to pay the paltry sum of a nickel for the self-same creations, and consummate origami was priced at 27 cents. The acute implication of this data is that builders without fundamental instruction, rote, or preparation imbue value into their products at comparable levels to the market rates for expert-crafted products.

At first glance, it may seem odd to evaluate the cognitive biases exploited in the creation of consumer goods; however, there are parallels to trends within the geospatial technology. The increasing prevalence of software as a service (SaaS), has shifted the focus of geospatial technology from desktop-based programs to distributed cloud-based applications. The consumerization of software users outside of the initial purchase and licensing is mostly achieved by aggressively offering online, custom-off-the-shelf (COTS) applications. The GIS professional’s interactions with these COTS applications mirrors the ready-to-assemble furniture experience; minimal instructions, standardized parts throughout an ecosystem, and lightweight components requiring no technical expertise. If cloud COTS applications mimic the flat-pack workflow, how do we make the most of our understanding of the IKEA effect?

Built to Last

The worst web maps I’ve ever had the pleasure of seeing were made by me, I have also built a spectacular end table that holds the litter box for my cat. The more honest truth is the cat-box seems to be coming apart at the hinges, and requires a weekly kick to straighten it out; my web applications suffer the most when I don’t take the time to polish the end-user experience. At home, my feline customer doesn’t complain about the architectural deficits introduced in the assembly of the facilities. At work, my potential customer base includes 180,000 residents, the working population and any visitors or guests that the City may see on a given day. The following are some selected strategies to employ in order to polish and more effectively utilize COTS products available in ArcGIS Online (AGOL).

One of the greatest strengths of the COTS paradigm is the inclusion of What-You-See-Is-What-You-Get (WYSIWYG) preview capabilities. These applications allow you to visualize what the end-user sees when interacting with your product. This is not a frill, it’s an opportunity to preempt embarrassing phone calls, emails, and social media posts from the viewer-base. The temptation to stand up an application, in mere minutes often sacrifices the chance to create a great experience for the end-user, one that is unencumbered by distractions created through a sloppy delivery.

Akin to reading the instructions before assembly, a standardized checklist should be an integral part of the development process before any COTS application is floated either internally or externally. Cartographic conventions aside, this usually starts with your Web Map item and configuring the pop-ups. From legacy attribute architecture to the 10 character limit of the dreaded shapefile, your <Field Alias> needs to be different from your <Field Name>; concise, explanatory, and use spaces. Don’t display attributes you don’t need, this includes ObjectID and shape fields. Get rid of the thousands separator option. Omit unnecessary pop-ups, if it’s the only thing on the map and has a unique symbology, this would be an excellent item to put into a legend.

Metadata is helpful to your colleagues, other GIS professionals, and your future self. There are now subtle prompts directing you to fill in metadata in the description page of your Web Mapping Application, Web Map, Dashboard, Feature Layer, etc. A summary should be at least 30 words, and accurately describe the purpose of your map. The description should be more than 100 words, and might go further into detail about the types of layers within the application, update schedules, links to other applications in the AGOL ecosystem, and other pertinent information. Tags will help others find your work, and thinking about how to tag your own work can lead you to other sources of inspiration for the problem you’re trying to tackle.

For the applications themselves, throw in your logos/branding, standardize the color scheme between applications and make sure that you link back to your main site. Your tagline doesn’t have to be “with Web AppBuilder for ArcGIS” or “A story map”. For the more narrative-driven applications, spelling and grammar become important considerations. While you may not be a wordsmith, there are plenty of free applications and browser plug-ins to ferret out the most major infractions; I’d recommend giving Grammarly² a go for basic spelling and grammar, or the Hemingway Editor³ if you suffer from being too rambling and verbose.

Despite the straightforwardness of the phrase, face-value is difficult to assess under the dual influences of the IKEA effect and the usual intimacy of a GIS professional with the data in which they work. Developing standardized ways of interacting with COTS applications will help ensure that the end-user experience is as uniform as possible between each visit and across differing applications hosted by your organization. At the end of the day, I shouldn’t have to kick the catbox, I should have built it correctly out-of-the-box. In like fashion, robust standard operating procedures should keep the wobbles out of your web apps.

Display with Pride

Interaction with the AGOL COTS ecosystem exposes us to the IKEA effect, but unlike the average consumer, our shoddy constructions are not safely sequestered within the halls of a private domicile, they live in the cloud, available to anyone across the planet with an internet connection. This makes it imperative that we identify and resolve our psychological predispositions to value work at a level higher than its face value. While we may gloss over the imperfections inherent in our own works, the end-user is unlikely to be as forgiving; our perceived value closer to a nickel than a quarter.

“Don’t try to get all fancy…it doesn’t work.”

The word “fancy” brings up about 20 results in my inbox, this is one of them. I’ve also gathered other gems like “interactive map is horrible” and “map is useless”. In all fairness, they were right. The map in question had an issue that made the selection radius horribly small, a handful of pixels. Less than noticeable on a desktop computer, glaringly obvious on any mobile device. This experience opened my eyes and let me see the unwieldy, rickety applications that I had proudly deployed.

This same critique can be applied to the ever venerable All-In-One map. If your map has 30 layers, you’re not doing anyone a service by hosting it. Disparate data needs to be presented with the proper context, single subject, focused applications should be the rule of thumb. In a similar vein, just because you can add a dashboard doesn’t mean that you should. Your data tells a story, it does so in specific ways. Averages and on-the-fly calculators don’t mean anything if the data they crunch was not significant to begin with.

I’ve heard it said a few different ways, usually a variation of, if X can’t use it, then you need to make it simpler. Be careful to casually cast a net involving X-type of person, the more accurate advice is to ask your audience to test your application. This is easiest if your audience is limited or exactly known. If your audience is nebulous or too broad, go and ask subject matter experts, GIS colleagues, laypersons, and those you would consider to be computer illiterate. Each of these people will provide insight that you may not have had yourself, or find difficult to see through the spectre of the IKEA effect.

The general trends of the geospatial industry are not changing anytime soon, and we will continue to see SaaS play an ever larger role as data complexity deepens and so too the computational requirements to effectively analyze it. The demand for cloud-based infrastructure, out-of-the-box data science, and COTS applications will only be outpaced by the avalanche of data that we continue to produce in real-time streams through increasingly web-enabled devices. These tools are powerful, convenient, but rely on “the builder” to assemble them correctly. Our natural impulse is to over-value the fruits of our labor, even when it involves minimal expertise. Recognizing this cognitive bias allows us to control its effects. A detailed and thoughtful examination of our interactions with ready-to-assemble applications can help us pass along showroom-quality experiences to “the others” that use our products the most.

¹Norton, Michael I., Daniel Mochon, and Dan Ariely. “The IKEA Effect: When Labor Leads to Love.” Journal of Consumer Psychology 22, no. 3 (July 2012): 453–460.

²www.grammarly.com

³www.hemingwayapp.com

Peculiarities of time

Albeit discouraging, thinking about GIS in terms of constraints helps us better assess the return on investment of any project that we undertake. The greatest constraint that any of us face in our professional life is time. In an idealized world, a GIS team can only spend approximately 40 hours per person from their productivity bank, most of which is already disbursed by taking on the many maintenance and upkeep tasks that keep a good GIS humming along. How do we maximize our weekly allowance of productive, actionable hours? What do we do in the face of a deficit?

Our City recently ran into a rather thorny issue that threatened to be a productivity bank-busting expenditure; we needed suite numbers. Some numeric context: in the past 30 years, our City has gone from a quaint County-seat of 20,000 to a bustling extension of the DFW metroplex with an estimated population of more than 175,000 and encompasses nearly 68 square miles. We issue an average of 40,000 permits per year and document more than 12,000 code enforcement cases. This year we are making the transition to an all-digital workflow; based off of a combination of our parcel layer and the more than 68,000 points that make up our address layer. A permit or case can’t be handed out without a unique address from our GIS. We needed suite numbers.

The first one to sound the warning klaxon was Mike, one of the managers of the Code department. An intimidating man who gets the job done by thinking through the details, his desire to do the job correctly is only outpaced by his fear that it might not get done at all. Like the genesis of many daunting tasks, I immediately felt my stomach churn and my anxieties run wild as I tried to comprehend the magnitude of the problem, calculate the steps to complete this request, and the requisite working hours that I would need to feed this data monster. Questions rushed through my internal monologue:

· How many actual? Approximate? What percent of our total addresses? · Does this data already exist somewhere? Who would have it? · Do we need to create it? How? What datasets can help us? · Where are they physically at?

Punctuated between these questions, I kept coming back to the phrase, “boots on the ground.” It’s direct, conveys the need for physical observation, and with the militaristic connotation sounds a bit heroic– –at least more heroic than City work is generally considered. Sans boots, we might have been able to grab some information to create a reasonably complete set of suites, but why create an inferior dataset?

The year is 2017. We have mobile devices, door-to-door routing, and a GIS capable of incredible accuracy. Why would we stack all the address points on top of each other? Why not put those points exactly where they go?, on the specific part of the building in question? Why burn a few minutes of each inspector’s time looking for where they’re going, when our GIS data (read through the new software) could route them right to the front door? It may have been faster in the short-term to do the job half-right, but in the long-term, I’d be directly responsible for slowly bleeding the productivity bank of another department through the micro-transactions of wasted time.

When spatial isn’t special

The GIS solution for this project was straight-forward enough to be featured on any homework assignment focusing on field verification: group points derived from matching primary address fields, grid index features, make data-driven pages, and print. For those of us using ESRI products, the next step would be to spin up a feature service with editable layers, and then to decide between ArcCollector or the Web App Builder for the input format. However, the more I thought through the workflow, the more I had a nagging feeling that I wasn’t going to be the one doing any of this work.

Sixty-five grid squares and 778 potential suite addresses later, I came to a realization: this project was too big for my current workload. Sure I could work through my lunch every day for the next few months, but there wasn’t any guarantee that I’d be able to get it done in time. I still thought that boots on the ground was the best way to complete this project, but I accepted that they wouldn’t be my own two boots.

Like many GIS professionals, I am often guilty of an obsession with novelty. The drone, the cloud, the next upgrade. Constantly chasing the horizon can cause us to neglect the wealth of resources that we have on hand– –the most important, human resources. While spatial workers may be in short supply in many organizations; field staff, especially those that participate in revenue generation, are more abundant.

The Code Enforcement Officers were all over the City, all day, every day. They were driving past these facilities, and at times, initiating cases involving them. These people know our geographic area in a way more intimate than our GIS staff; forgoing bird’s eye for eye-level. Who better to empower than the people most familiar with the City?

Another mistake common to our trade is to assume that what we do is some sort of techno-wizardry. The spatial skillset is a framework that serves as a force multiplier for organizations. We should be enabling the best practices in our respective organizations; less Atlas, more Prometheus; less author, more printing press; less autocratic, more pragmatic.

Understandably, this is easier said than done. Many GIS professionals are accustomed to project stewardship from inception to completion. Giving up control requires trusting other members within your organization; more importantly, it requires an admission that a GIS person is not necessarily required to perform GIS work. The sooner that these requirements are realized, the sooner your GIS productivity bank can decrease non-critical expenditures and increase your weekly allowance of actionable hours.

Flexibility

My plan was elegant, sweeping, and utterly dismissive of how Code Enforcement conducts their business. I had started my GIS career as part of a consulting firm, and as any good consultant is wont to do, I scheduled a presentation to show off the deliverables. 36x36 inch maps, printed excel sheets, and a suit jacket with khakis– –for consultants, it’s a thing.

I had just finished presenting my plan, each Code Enforcement Officer would spend approximately one hour a week investigating address sites. Documentation would be simple. Literally pen and paper. These pages then sent via inter-office mail to the GIS team that handled addressing.

In his usual no-nonsense style, Code Enforcement Mike told me, yeah that’s just not possible.

I thought a 3% time expenditure across the board was modest payment for such a large initiative. Because this wasn’t my department, and I was still fairly new to the City, I carefully replied, that I didn’t have a proverbial dog in the fight. My interests lay in making sure Code got what they needed.

Mike pointed out, this is our busiest time of year. Our guys literally don’t have any time to spare.

I hadn’t considered this. Code cases pile up in the summer. When the weather’s warm, people come outside, the grass grows, and violations abound. Our metrics are tight. All calls responded to within a day. Mike’s team never missed this goal. Their productivity bank didn’t have a moment to spare.

But I’m not saying we can’t do it. We’ve got this guy, Briggs. We hired him part-time to take care of the mosquito trap workflow and removing nuisance signage.

The urgency behind the much-publicized Zika virus outbreaks last year had us all braced for the worst. It was forward-thinking to request additional staff for this year’s mosquito season; re-prioritizing Briggs’ schedule reduced their bandwidth, but it made the address suites mapping possible.

Mike finished vouching, this way, everything’s going to be done the exact same way.

I agreed with Mike’s assessment. On first glance, we might be losing speed, but we’d be gaining standardization; one submission format would increase the accuracy in translating the paper observations to the GIS by the addressing team.

I’m hanging this map up on my wall and we’re going to mark off each square as it comes in. I’ll keep you posted. With that, we shook hands and I returned to my office.

Added Benefits

For the next two months, Mike’s guy, Briggs diligently mapped all the address suites throughout the City. In the end, the work was less than heroic, and it so happened that there were only two boots on the ground. In the peak heat of the summer his boots walked retail centers, his boots went into office buildings. Briggs wasn’t self-taught GIS, he never had a university course. He did his job, he drove the City, and he wrote down what he saw. At the end of the project, he had visited 218 buildings and recorded 1,362 suites.

Possession of a completed unique addressing layer let us continue business as usual. We met the minimum requirements for the new software. That’s good. Now, however, the increased granularity of our newly augmented address suites dataset allows door-to-door routing for emergency services to any location in the City. That’s great. Because for these crews, the micro-transactions of wasted time can easily mean lives lost. For them, Briggs shoring up their productivity bank is the definition of heroic. I just showed up with some maps.