Public Member Functions
void	BaseTest_Cleanup ()

void	BaseTest_Initialize ()

virtual void	Test_API_Detect_BadAPITest ()

void	Test_API_PollUntilReady_TimeoutException ()

virtual void	Test_API_Seed_And_SeedCleanup ()

void	Test_Concurrent_Resource_Injection ()

void	Test_Reflection_RandomizedReference_ValidationEnabled ()
	Q: Why do these test matter?

Static Public Member Functions
static void	ApplyConfiguredCredentials ()
	Apply the current configured API_URL and API_AuthenticationToken credentials to the static API.

static void	CheckConnection ()
	Set the Client API Server URL to the configured API_URL.

static void	TestSuiteCleanup ()
	Actions to run once when the C# Unit Tests finish. Note: This method will not automatically be inherited and run by tests suites in different assemblies that import this library. You will need to create your own method with the [AssemblyCleanup] attribute that calls this method.

static void	TestSuiteInitialize (TestContext context)
	Actions to run once when the C# Unit Tests initialize. Note: This method will not automatically be inherited and run by tests suites in different assemblies that import this library. You will need to create your own method with the [AssemblyInitialize] attribute that calls this method.

Static Public Attributes
static Reflection	Reflection
	Shared instance of a class for generating random resources.

static Samples	Samples
	Shared instance of a class for generating reusable sample resources.

Protected Member Functions
void	AddCommonTestCleanupAction (Action action)

void	AddCommonTestInitializeAction (Action action)

Static Protected Member Functions
static void	BaseClass_Cleanup (Action action)

static void	BaseClass_Initialize (TestContext context, Action action)

Properties
static IAccessToken	API_AuthenticationToken `[get, set]`
	Set the AuthenticationToken to supply to the server before every test (if applicable).

static string	API_URL `[get, set]`
	Location of the REST API being tested against. This can be set using the API_URL environment variable. Default is https://localhost:8000/.

bool	DeletingResourcesBetweenTests `[get]`
	Determines based on the EnvironmentSettings whether injected resources (such as Samples) are being deleted between each test.

TestContext	TestContext `[get, set]`

Detailed Description

Definition at line 31 of file Meta_TestSuite_Utilities_Server.cs.

Inheritance diagram for AnalyzeReTesting.Tests.Utilities.Meta_TestSuite_Utilities_Server:

Member Function Documentation

◆ AddCommonTestCleanupAction()

void AnalyzeReTesting.Base.BaseTestSuite.AddCommonTestCleanupAction ( Action action )

inlineprotectedinherited

Definition at line 80 of file BaseTestSuite.cs.

◆ AddCommonTestInitializeAction()

void AnalyzeReTesting.Base.BaseTestSuite.AddCommonTestInitializeAction ( Action action )

inlineprotectedinherited

Definition at line 75 of file BaseTestSuite.cs.

◆ ApplyConfiguredCredentials()

static void AnalyzeReTesting.Base.BaseServerTestSuite.ApplyConfiguredCredentials ( )

inlinestaticinherited

Apply the current configured API_URL and API_AuthenticationToken credentials to the static API.

Definition at line 226 of file BaseServerAPITestSuite.cs.

◆ BaseClass_Cleanup()

static void AnalyzeReTesting.Base.BaseTestSuite.BaseClass_Cleanup ( Action action )

inlinestaticprotectedinherited

Definition at line 66 of file BaseTestSuite.cs.

◆ BaseClass_Initialize()

static void AnalyzeReTesting.Base.BaseTestSuite.BaseClass_Initialize	(	TestContext	context,
		Action	action
	)

inlinestaticprotectedinherited

Definition at line 56 of file BaseTestSuite.cs.

◆ BaseTest_Cleanup()

void AnalyzeReTesting.Base.BaseTestSuite.BaseTest_Cleanup ( )

inlineinherited

Definition at line 46 of file BaseTestSuite.cs.

◆ BaseTest_Initialize()

void AnalyzeReTesting.Base.BaseTestSuite.BaseTest_Initialize ( )

inlineinherited

Definition at line 35 of file BaseTestSuite.cs.

◆ CheckConnection()

static void AnalyzeReTesting.Base.BaseServerTestSuite.CheckConnection ( )

inlinestaticinherited

Set the Client API Server URL to the configured API_URL.

Definition at line 235 of file BaseServerAPITestSuite.cs.

◆ Test_API_Detect_BadAPITest()

virtual void AnalyzeReTesting.Tests.Utilities.Meta_TestSuite_Utilities_Server.Test_API_Detect_BadAPITest ( )

inlinevirtual

Definition at line 237 of file Meta_TestSuite_Utilities_Server.cs.

◆ Test_API_PollUntilReady_TimeoutException()

void AnalyzeReTesting.Tests.Utilities.Meta_TestSuite_Utilities_Server.Test_API_PollUntilReady_TimeoutException ( )

inline

Definition at line 263 of file Meta_TestSuite_Utilities_Server.cs.

◆ Test_API_Seed_And_SeedCleanup()

virtual void AnalyzeReTesting.Tests.Utilities.Meta_TestSuite_Utilities_Server.Test_API_Seed_And_SeedCleanup ( )

inlinevirtual

Definition at line 249 of file Meta_TestSuite_Utilities_Server.cs.

◆ Test_Concurrent_Resource_Injection()

void AnalyzeReTesting.Tests.Utilities.Meta_TestSuite_Utilities_Server.Test_Concurrent_Resource_Injection ( )

inline

Definition at line 37 of file Meta_TestSuite_Utilities_Server.cs.

◆ Test_Reflection_RandomizedReference_ValidationEnabled()

void AnalyzeReTesting.Tests.Utilities.Meta_TestSuite_Utilities_Server.Test_Reflection_RandomizedReference_ValidationEnabled ( )

inline

Q: Why do these test matter?

A: Input fuzzing is an extremely important part of integration tests. Our unit test cases throughout the system are laughably simple, and clients build incredibly convoluted structures in production, using resources in combinations that we not only don't test for, but often times didn't consider when implementing new features with only the simplest use-cases in mind.

Randomized or "input-fuzzing" tests have been responsible for catching dozens if not hundreds of test cases in the past where the server either locked up indefinitely, blew up unceremoniously, or otherwise behaved unpredictably in the face of special combinations of structures and metadata values. While many of these discoveries have been deemed low priority and gone unaddressed (search for ARE tickets labeled "Validation"), many have discovered real test cases that needed to be accounted for, or otherwise revealed core issues in the way the system handles errors at various layers (e.g. API/AE/SE).

Each "unique" invalid combination discovered leads to added complexity in the form of various Reflection method tests to capture (often undocumented) expectations and behaviour of our API. While some of this behaviour is necessary, often it leads to requiring new validation and/or logic in client-facing interfaces (like the API and Excel) in order to prevent the user from making a mistake that will only show up much later in their workflow.

As such, to disable this or related tests, or to cripple its ability to "randomize" property values in an overly-restrictive way, will end an era of carefully considering how users interact with the system, and lead to many production issues surfacing in difficult-to-diagnose ways weeks, months, or even years down the line, as clients are often slow to adopt new features into their workflows. If and when these issues are discovered down the line, they will erode at the confidence the customer is meant to have in our system, something which cannot simply be "patched" as needed.

Definition at line 116 of file Meta_TestSuite_Utilities_Server.cs.

◆ TestSuiteCleanup()

static void AnalyzeReTesting.Base.BaseServerTestSuite.TestSuiteCleanup ( )

inlinestaticinherited

Actions to run once when the C# Unit Tests finish. Note: This method will not automatically be inherited and run by tests suites in different assemblies that import this library. You will need to create your own method with the [AssemblyCleanup] attribute that calls this method.

Definition at line 163 of file BaseServerAPITestSuite.cs.

◆ TestSuiteInitialize()

static void AnalyzeReTesting.Base.BaseServerTestSuite.TestSuiteInitialize ( TestContext context )

inlinestaticinherited

Actions to run once when the C# Unit Tests initialize. Note: This method will not automatically be inherited and run by tests suites in different assemblies that import this library. You will need to create your own method with the [AssemblyInitialize] attribute that calls this method.

Definition at line 84 of file BaseServerAPITestSuite.cs.

Member Data Documentation

◆ Reflection

Reflection AnalyzeReTesting.Base.BaseClientTestSuite.Reflection

staticinherited

Shared instance of a class for generating random resources.

Definition at line 18 of file BaseClientTestSuite.cs.

◆ Samples

Samples AnalyzeReTesting.Base.BaseClientTestSuite.Samples

staticinherited

Shared instance of a class for generating reusable sample resources.

Definition at line 15 of file BaseClientTestSuite.cs.

Property Documentation

◆ API_AuthenticationToken

IAccessToken AnalyzeReTesting.Base.BaseServerTestSuite.API_AuthenticationToken

staticgetsetinherited

Set the AuthenticationToken to supply to the server before every test (if applicable).

Definition at line 32 of file BaseServerAPITestSuite.cs.

◆ API_URL

string AnalyzeReTesting.Base.BaseServerTestSuite.API_URL

staticgetsetinherited

Location of the REST API being tested against. This can be set using the API_URL environment variable. Default is https://localhost:8000/.

Definition at line 28 of file BaseServerAPITestSuite.cs.

◆ DeletingResourcesBetweenTests

bool AnalyzeReTesting.Base.BaseServerTestSuite.DeletingResourcesBetweenTests

getinherited

Determines based on the EnvironmentSettings whether injected resources (such as Samples) are being deleted between each test.

Definition at line 46 of file BaseServerAPITestSuite.cs.

◆ TestContext

TestContext AnalyzeReTesting.Base.BaseTestSuite.TestContext

getsetinherited

Definition at line 20 of file BaseTestSuite.cs.

The documentation for this class was generated from the following file:

Meta_TestSuite_Utilities_Server.cs

Public Member Functions

Static Public Member Functions

Static Public Attributes

Protected Member Functions

Static Protected Member Functions

Properties

Detailed Description

Member Function Documentation

◆ AddCommonTestCleanupAction()

◆ AddCommonTestInitializeAction()

◆ ApplyConfiguredCredentials()

◆ BaseClass_Cleanup()

◆ BaseClass_Initialize()

◆ BaseTest_Cleanup()

◆ BaseTest_Initialize()

◆ CheckConnection()

◆ Test_API_Detect_BadAPITest()

◆ Test_API_PollUntilReady_TimeoutException()

◆ Test_API_Seed_And_SeedCleanup()

◆ Test_Concurrent_Resource_Injection()

◆ Test_Reflection_RandomizedReference_ValidationEnabled()

◆ TestSuiteCleanup()

◆ TestSuiteInitialize()

Member Data Documentation

◆ Reflection

◆ Samples

Property Documentation

◆ API_AuthenticationToken

◆ API_URL

◆ DeletingResourcesBetweenTests

◆ TestContext