Users get disconnected during test runs

Question

I work on a Ruby on Rails app that has many test in its deployment process (~3000) We use capybara and selenium for feature tests. We recently migrated CI from Heroku CI to CircleCi. We run 10 docker instances to run our test suite.

For many feature tests, we use this block of code to sign in admin users:

def admin_signs_in
  admin_user = FactoryBot.create(:admin_user, :for_signin)
  sign_in admin_user
  return admin_user
end

But sometimes, randomly, one test doesn't pass because users get disconnected and cannot access to the page needed for the test.

An example of failing test: require 'rails_helper'

RSpec.describe "Admin sets client budget to project", js: true do
  let(:client)  { create(:client) }
  let(:project) { create(:project, client: client) }
  let(:timeslots_proposal) { create(:timeslots_proposal_with_client, project: project) }

  before do
    admin_signs_in
    # this path needs an authenticated user
    visit advanced_admin_timeslots_proposal_path(timeslots_proposal)

    within "#client-budget" do
      find(".fa-pen").click
    end
  end

  describe "negative amount" do
    before do
      within "#some_container" do  
        # this block doesn't fail, meaning that at this point, user is authenticated          
        expect(find("#some_field")["placeholder"]).to eq "Enter amount"
        fill_in "some_field", with: "-235.99"
        find('.btn-primary').click
      end
    end

    it "raises an error" do
      # this fails, logs indicate that user is redirected to signin page
      expect(page).to have_content "Amount must be positive"
    end
  end
end

We tried to use rspec-retry gem to try those tests several times, but when it fails once, it fails for all retries.

It can happen to any test in our suite, I'd say it happens in 0.1% of authenticated tests, randomly.

I suspect a problem with Capybara cookies, but I don't know how to solve this.

Answer 1

Use np.argwhere and random.choice:

import numpy as np
import random

a = np.random.choice([0, 1, 2], (20, 20))
i = random.choice(np.argwhere(a == 1))

Answer 2

You can use numpy.where to identify the coordinates of the cells of value 1, then pick one (or many) random indices from this output:

# test array
a = np.random.randint(0, 3, size=(15, 10))

# get coordinates where a == 1
coords = np.c_[np.where(a==1)]

# pick N random cells
N = 1
out = coords[np.random.choice(coords.shape[0], N, replace=False)]

# check
x,y = out[0]
assert a[x,y] == 1

example: (5, 3)

a:

array([[0, 1, 0, 1, 1, 2, 0, 2, 0, 0],
       [0, 2, 1, 2, 2, 0, 1, 1, 1, 1],
       [0, 1, 0, 0, 1, 2, 0, 2, 0, 1],
       [1, 2, 0, 1, 1, 1, 0, 2, 0, 2],
       [2, 0, 2, 0, 0, 0, 1, 1, 2, 0],
       [0, 1, 0, 1, 2, 2, 0, 1, 1, 1],
       [1, 2, 2, 2, 0, 2, 1, 0, 1, 2],
       [0, 0, 2, 0, 0, 0, 0, 0, 0, 2],
       [0, 2, 1, 1, 1, 0, 1, 1, 1, 0],
       [1, 2, 0, 1, 2, 0, 2, 0, 1, 2],
       [2, 1, 0, 1, 1, 0, 2, 2, 2, 2],
       [1, 2, 2, 2, 2, 2, 0, 1, 2, 2],
       [1, 2, 1, 0, 2, 2, 0, 2, 0, 0],
       [2, 0, 2, 2, 2, 0, 0, 0, 1, 2],
       [0, 1, 2, 2, 2, 1, 0, 0, 0, 0]])