Azure Data Factory - Dynamic Skip Lines Expression

Question

I am attempting to import a CSV into ADF however the file header is not the first line of the file. It is dynamic therefore I need to match it based on the first column (e.g "TestID,") which is a string.

Example Data (Header is on Line 4)

Date:,01/05/2022
Time:,00:30:25
Test Temperature:,25C
TestID,StartTime,EndTime,Result
TID12345-01,00:45:30,00:47:12,Pass
TID12345-02,00:46:50,00:49:12,Fail
TID12345-03,00:48:20,00:52:17,Pass
TID12345-04,00:49:12,00:49:45,Pass
TID12345-05,00:50:22,00:51:55,Fail

I found this article which addresses this issue however I am struggling to rewrite the expression from using an integer to using a string.

https://kromerbigdata.com/2019/09/28/adf-dynamic-skip-lines-find-data-with-variable-headers

First Expression

iif(!isNull(toInteger(left(toString(byPosition(1)),1))),toInteger(rownum),toInteger(0))

As the article states, this expression looks at the first character of each row and if it is an integer it will return the row number (rownum)

How do I perform this action for a string (e.g "TestID,")

Many Thanks

Jonny

Answer 1

I think you want to consider first line that starts with string as your header and preceding lines that starts with numbers should not be considered as header. You can use isNan function to check if the first character is Not a number(i.e. string) as seen in the below modified expression:

iif(isNan(left(toString(byPosition(1)),1))
    ,toInteger(rownum)
    ,toInteger(0)
   )

Following is a breakdown of the above expression:

1. left(toString(byPosition(1)),1): gets first character fron left side of the first column.
2. isNan: checks if the character is "not a number".
3. iif: not a number, true then return rownum, false then return 0.

Or you can also use functions like isInteger() to check if the first character is an integer or not and perform actions accordingly.

Later on as explained in the cited article you need to find minimum rownum to skip.

Hope it helps.

Answer 2

Okay, what you need to do here is to create the circle from 3 positions.

let counter = -1;
const positions = [
    [5, 0, 5],
    [0, 0, 5.66],
    [-5, 0, 0]
];
const cubes = [i1, i2, i3];

With the counter, whenever you click on the button, the counter will increase and we will have something like:

• 1st click: counter=0 => 0, 1, 2
• 2nd click: counter=1 => 1, 2, 0
• 3rd click: counter=2 => 2, 0, 1
• 4th click: counter=0 => 0, 1, 2

and so on, you see, it loops, and [0, 1, 2] is the index of the position (x,y,z) that you will apply to the cube. On the 4th click when the counter is bigger than the total of positions, we set it back to the beginning.

This is the new rotWorld() function:

function rotWorld(){
    console.log('--- rotate world');

    // counter store the begin of position loop
    counter++;
    counter = counter < positions.length ? counter : 0;

    // loop through each cube
    cubes.forEach((cube, index) => {
        // get the position base on the counter
        const pos = (counter + index) % positions.length;
        console.log(pos);
        gsap.to(cube.position, {
            duration: 1,
            x: positions[pos][0],
            y: positions[pos][1],
            z: positions[pos][2],
        });
    });
}

I added some log in the console so you can test it out. You can also reverse the direction using the same logic. See Fiddle