Scoring

There are 3 major components of the Trial Score.

Cost Factor: How much does the system (sensors) cost?

Efficiency Factor: How fast or efficiently did the system complete the task(s)

Task Score: How well did the task(s) get performed?

Cost Factor

The Cost Factor $C F$ compares the cost of the sensors chosen by the team to the average of all sensor configurations across all teams.

$T C$ is the total cost of the sensors in the team’s configuration.

$T C_{a v g}$ is the average sensor cost across all teams.

$w_{c}$ is a weighting constant for cost factor.

Cost Factor

C F = w_{c} \cdot \frac{T C_{a v g}}{T C}

Efficiency Factor

The Efficiency Factor $E F_{i}$ for order $i$ compares the time to complete order $i$ for the team to the average of all teams’s times to complete order $i$ .

$T_{i}$ is the time to complete order $i$

$T_{a v g_{i}}$ is the average time to complete order $i$ for all teams

$w_{t}$ is a weighting constant for efficiency factor.

Efficiency Factor

E F_{i} = w_{t} \cdot \frac{T_{a v g_{i}}}{T_{i}}

Task Scores

Each submitted order will recieve a task score based on the required task for that order (Kitting, Assembly, or Combined). Each task score is generated from Boolean conditions.

Kitting Task Score

A kitting task has $n$ parts that need to be placed on the kitting tray.

A shipment has $m$ parts on the kitting tray.

For each task there are two Boolean conditions:

$isCorrectTrayID \to A$ : true if the shipment tray ID is correct.

$isCorrectDestination \to B$ : true if the shipment was sent to the correct destination.

For each quadrant $q$ of the kitting tray there are four Boolean conditions:

${isCorrectType}_{q} \to C$ : true if the part type in quadrant $q$ is correct.

${isCorrectColor}_{q} \to D$ : true if the part color in quadrant $q$ is correct.

${isFlipped}_{q} \to E$ : true if the part in quadrant $q$ is flipped.

${isFaulty}_{q} \to F$ : true if the part in quadrant $q$ is faulty.

Tray Points

\begin{array}{r} {pt}_{t} = {\begin{cases} 1, & if A \\ 0, & otherwise \end{cases} \end{array}

Quadrant Points

\begin{array}{r} {pt}_{q} = {\begin{cases} 0, & if \neg C \lor E \lor F \\ 3, & if D \\ 2, & if \neg D \end{cases} \end{array}

Bonus Points

\begin{array}{r} {pt}_{b} = {\begin{cases} n, & if \sum_{q}^{n} {pt}_{q} = n \times 3 \\ 0, & otherwise \end{cases} \end{array}

Extra Parts Penalty

A penalty is only applied if more parts are on the tray than needed.

\begin{array}{r} {pn}_{e p} = {\begin{cases} m - n, & if m > n \\ 0, & otherwise \end{cases} \end{array}

Wrong Tray Penalty

\begin{array}{r} {pn}_{t} = {\begin{cases} 1, & if \neg A \\ 0, & otherwise \end{cases} \end{array}

Destination Multiplier

\begin{array}{r} {pm}_{d} = {\begin{cases} 1, & if B \\ 0, & otherwise \end{cases} \end{array}

Kitting Task Score

S_{k} = ({pt}_{t} + \sum_{q}^{n} ({pt}_{q}) + {pt}_{b} - {pn}_{e p} - {pn}_{t}) \times {pm}_{d}

The task score cannot be negative, if the calculation is negative the score will be set as 0.

Assembly Task Score

An assembly task has $n$ parts that need to be assembled into the insert.

For each task there is one Boolean condition:

$isCorrectStation \to A$ : true if the assembly was done at the correct station (as1, as2, as3, or as4).

Each slot $s$ in the insert has the following Boolean conditions:

${isAssembled}_{s} \to B$ : true if the part in slot $s$ is reported as assembled.

${isCorrectColor}_{s} \to C$ : true if the part in slot $s$ is of correct color.

${isCorrectPose}_{s} \to D$ : true if the part in slot $s$ has the correct pose.

Slot Points

\begin{array}{r} {pt}_{s} = {\begin{cases} 0, & if \neg B \\ 3, & if C \land D \\ 2, & if C \lor D \\ 1, & if \neg C \land \neg D \end{cases} \end{array}

Bonus Points

\begin{array}{r} {pt}_{b} = {\begin{cases} n, & if \sum_{s}^{n} {pt}_{s} = n \times 3 \\ 0, & otherwise \end{cases} \end{array}

Station Multiplier

\begin{array}{r} {pm}_{s} = {\begin{cases} 1, & if A \\ 0, & otherwise \end{cases} \end{array}

Assembly Task Score

S_{a} = (\sum_{s}^{n} {pt}_{s} + {pt}_{b}) \times {pm}_{s}

Combined Task Score

A combined task has $n$ parts that need to be gathered from the environment and assembled to the insert.

For each task there is one Boolean condition:

$isCorrectStation \to A$ : true if the assembly was done at the correct station (as1, as2, as3, or as4).

Each slot $s$ in the insert has the following Boolean conditions:

${isAssembled}_{s} \to B$ : true if the part in slot $s$ is reported as assembled.

${isCorrectColor}_{s} \to C$ : true if the part in slot $s$ is of correct color.

${isCorrectPose}_{s} \to D$ : true if the part in slot $s$ has the correct pose.

Slot Points

\begin{array}{r} {pt}_{s} = {\begin{cases} 0, & if \neg B \\ 5, & if C \land D \\ 4, & if C \lor D \\ 3, & if \neg C \land \neg D \end{cases} \end{array}

Bonus Points

\begin{array}{r} {pt}_{b} = {\begin{cases} n, & if \sum_{s}^{n} {pt}_{s} = n \times 5 \\ 0, & otherwise \end{cases} \end{array}

Station Multiplier

\begin{array}{r} {pm}_{s} = {\begin{cases} 1, & if A \\ 0, & otherwise \end{cases} \end{array}

Combined Task Score

S_{c} = (\sum_{s}^{n} {pt}_{s} + {pt}_{b}) \times {pm}_{s}

Trial Score

The trial score $T S$ combines the cost factor, efficiency factors and task scores into a single score for ranking the teams.

For each order there is one Boolean condition:
- $isPriorityOrder \to A$ : true if the order is classified as a priority order

Priority Multiplier

\begin{array}{r} {pm}_{p} = {\begin{cases} 3, & if A \\ 1, & otherwise \end{cases} \end{array}

Trial Score

T S = C F \times \sum_{i = 0}^{n} ({pm}_{p} \times E F_{i} \times S_{i})

Scoring

Cost Factor

Efficiency Factor

Task Scores

Kitting Task Score

Assembly Task Score

Combined Task Score

Trial Score

Versions

GitHub