Improve running end-to-end tests locally

[NBardelot]

• Allow users to run end-to-end tests with a fully-qualified and/or private alpine image
• Allow users to run end-to-end tests behind a proxy
• Introduces a new NO_PROXY parameter in the Makefile that is propagated alongside HTTP_PROXY and HTTPS_PROXY
• Introduces a new LIST_OF_E2E_TESTS parameter in the Makefile that drives the names of the tests passed to ./test_e2e.sh to run a selection of tests, running all tests by default
• Allow users to override the git-sync repository URL used by end-to-end tests, by exporting GIT_SYNC_REPOSITORY_URL in their environment, in order to use a mirror if need be
• Sort .gitgnore for convenience for future diffs
• Add .idea/ to .gitignore

NOTE : I propose this in parallel of the feature request #970 as I had to make some adaptations locally to be able to run the end-to-end tests and I think they could be used by others. It completes the PR #971 that has already been merged.

/kind feature

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: NBardelot
Once this PR has been reviewed and has the lgtm label, please assign thockin for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[thockin]

I appreciate the PR, but I don't think we want this. The argument to test_e2e.sh is a pattern, not just a list, and besides that, you can just run the script. It's easier than passing a named argument to make.

[NBardelot]

You can just run the script. It's easier than passing a named argument to make.

Cf. my comment in the test_e2e.sh script, the script itself makes a call to make without any possibility to override the makefile's extra parameters (for buildx, or to use fully-qualified images). Thus, you cannot run test_e2e.sh alone in the same conditions you build the rest of the project.

Running a subset of tests directly from make is the easiest workaround, with no refactoring needed. In an ideal world though, the make calls in test_e2e.sh would probably better be refactored out of the script itself, and in the makefile as a dependency goal.

By the way, you might already know of bats-core as a shell test framework, but if you don't it could be of interest to you, given it offloads a lot of the mechanisms you have to maintain in the test_e2e.sh. It could also help split the tests in smaller test suits, easier to manage for a newcomer like me :)

[thockin]

Somehow I only saw part of this PR when I looked at it before.

I learned of bats well after I did all the work, so didn't bother to convert it.

[thockin]

why not pass the important args?

[NBardelot]

At first glance I was surpised that the script was running its prerequisites by calling the Makefile. And I ended up calling the given goals ahead of time myself, and finding it simpler. The alternative would be to map 1..1 every Makefile arg as an env variable in the script, which is possible but not very practical.

[thockin]

Does this do anything? Each of these blocks runs in its own shell, so export has no meaning:

thockin@thockin-glaptop4:/tmp/m$ cat Makefile 
foo:
	@export FOO=bar
	@echo $$FOO

bar:
	@export FOO=bar; \
	echo $$FOO
thockin@thockin-glaptop4:/tmp/m$ make foo

thockin@thockin-glaptop4:/tmp/m$ make bar
bar

[NBardelot]

My bad. Let me add a commit to fix this.

I think someone who uses a proxy will have all three variables already configured in a dotenv file or profile, expecting the NO_PROXY standard variable to be passed alongside the two others. It looks strange when this one is not. So the goal here was to be consistent (and not having to debug why something will be routed through the proxy when the user's config contains an exclusion, for example: the private/local images registries, or the github project's local mirror).

[thockin]

Can I get something like:

local default_repo="https://github.com/kubernetes/git-sync"
local repo="${REMOTE_TEST_REPO_URL:-$default_repo}"

[thockin]

clarify that the values here are regexes against the testcase names

[thockin]

does make pass "" here if the list is empty? I think not, but I am not sure

[NBardelot]

Without quotes no argument is passed.

Example:

printargs.sh

#!/bin/bash
for a in "$@"; do
  echo "Arg: >$a<"
done

> ./printargs.sh foo $not_declared bar $not_declared
Arg: >foo<
Arg: >bar<

vs.

> ./printargs.sh foo "$not_declared" bar "$not_declared"
Arg: >foo<
Arg: ><
Arg: >bar<
Arg: ><