Skip to end of metadata
Go to start of metadata

Overview

The undo/redo feature test is used to test the undo/redo system in an automated way, reducing the regressions introduced by new changes.

The way this is done is by reproducing critical test cases with a set of python scripts. The scripts writer can specify a list of objects of objects and assets that need testing. This means that before and after a single test, the serialized state of an object/asset must be the same.

Serialization

  • Objects
    Each <> represents a node; <Object> is a node tag, the rest of the key-data pairs inside a node are the Node Attributes
    Nodes in more indented levels are children of the nodes in less indented levels.

The serialization of objects is carried out automatically and their state is saved in an .xml file structured like the following:

<Object Type="Group" Layer="Main" LayerGUID="656e7504-5d5c-d3f5-6ac4-59a5b87b9765" Id="77a40a99-428b-b304-cbd9-efaaacba4516" Name="Group-1" Pos="523.25,501.5,32" Rotate="1,0,0,0" Scale="1,1,1" ColorRGB="65280" UseCustomLevelLayerColor="0" MinSpec="0" Opened="1">
<Objects>
<Object Type="Brush" Layer="Main" LayerGUID="656e7504-5d5c-d3f5-6ac4-59a5b87b9765" Id="ae7e377b-928d-dfaa-5c2f-3fac5acbf4df" Name="primitive_cube-1" Parent="77a40a99-428b-b304-cbd9-efaaacba4516" Pos="-1.5,0,0" Rotate="1,0,0,0" Scale="1,1,1" ColorRGB="16777215"           UseCustomLevelLayerColor="0" MinSpec="0" MatLayersMask="0" Prefab="objects/default/primitive_cube.cgf" IgnoreVisareas="0" CastShadowMaps="1" GIMode="1" RainOccluder="1" SupportSecondVisarea="0" DynamicDistanceShadows="0" Hideable="0" LodRatio="100" ViewDistRatio="100"   ExcludeFromNavigation="0" NoDynamicWater="0" AIRadius="-1" NoStaticDecals="0" RecvWind="0" Occluder="0" DrawLast="0" ShadowLodBias="0" IgnoreTerrainLayerBlend="0" IgnoreDecalBlend="0" RndFlags="180060000408">
 <CollisionFiltering>
 <Type collision_class_terrain="0" collision_class_wheeled="0" collision_class_living="0" collision_class_articulated="0" collision_class_soft="0" collision_class_particle="0" gcc_player_capsule="0" gcc_player_body="0" gcc_vehicle="0" gcc_large_kickable="0" gcc_ragdoll="0" gcc_rigid="0" gcc_vtol="0"  gcc_ai="0"/>
 <Ignore collision_class_terrain="0" collision_class_wheeled="0" collision_class_living="0" collision_class_articulated="0" collision_class_soft="0" collision_class_particle="0" gcc_player_capsule="0" gcc_player_body="0" gcc_vehicle="0" gcc_large_kickable="0" gcc_ragdoll="0" gcc_rigid="0" gcc_vtol="0"  gcc_ai="0"/>
</CollisionFiltering>
</Object>
</Objects>
</Object>

This is a group with one brush object as a child.

  • Assets
    The main asset comparison is done via a Cyclic Redundancy Check (CRC) algorithm; moreover, to help with error logging, all asset metadata files (e.g .cryasset files) are serialized as .xml archives in memory.

Testing

The undo system supports many different kinds of undos such as undo group detach/attach, prefab extract all, etc, with different setup conditions. For this reason, the actions that are required to set up and execute a single test and its components are exposed via python scripts. This way it is possible to expose editor actions via a flexible and simple scripting language and test Sandbox python APIs at the same time.

All testing scripts reside in <Engine installation directory>/Editor/Scripts/Test/UndoRedo, and a scripts is composed of:

  • a setup() function to prepares the test, instantiate all necessary objects/assets and to add them to the objects_to_test/assets_to_test  lists,
  • an executeTest() function to execute all the test actions,
  • a cleanup() function to delete all objects created for this test,
  • an objects_to_test list (implicitly defined in the C++ APIs for each script) that has to be filled by the script writer with the names of objects to be tested,
  • an assets_to_test list (implicitly defined in the C++ APIs for each script) that has to be filled by the script writer with the guids of assets to be tested,

Also a test_common module with some utility functions to create and operate on prefab hierarchies is available to the test scripts to make test creation faster.

The testing system executes all .py scripts found in the Test/UndoRedo folder the following way:

  1. The objects_to_test and assets_to_test lists are created in the C++ side of the testing framework and made available to each test;
  2. It calls setup() function;
  3. All the objects added to the objects_to_test list are serialized in .xml archives, each with a root tag called <object name>-PreTest;

    The serialization takes place on the C++ side of the framework and an .xml archive is created in memory for each object to be tested.
  4. All the assets added to the assets_to_test list have all their file CRCs computed and their content stored as .xml archives when possible;
  5. It calls executeTest() function;
  6. All the objects added to the objects_to_test list are serialized in XML archives, each with a root tag called <object name>-PostTest;
  7. All the assets added to the assets_to_test list have all their file CRCs computed and their content stored as XML archives when possible;
  8. The list of archives serialized before executeTest are compared by the order they are added to the objects_to_test list with the archives serialized after executeTest;
  9. The list of asset files serialized before executeTest are compared by their CRCs by the order they are added to the assets_to_test list with the assets serialized after executeTest. In case CRC comparison fails, the asset files XML archives are also compared and the results are relayed to the user;
  10. Calls cleanup() function.

All Phyton scripts can be placed in the folder <Engine installation directory>/Editor/Scripts/Test.

A test named Editor/PythonTestUndoRedo on the Test Runner can be used to execute all the tests in this folder. Note that if there is no level open, a new one named PythonUndoTestsLevel will be automatically created to run the tests in (currently no deletion is possible because of sandbox limitations).

The Test Runner can be accessed via Tools → Advanced → Test Runner in the Sandbox menu.
For more information about Test Runner, please refer to the Test Runner page.

Custom Test Example

Following is a commented test script example that attaches an object to a group, undoes this and checks if both actions have been performed successfully.

This example can be used as a reference when a custom test is being created:

from sandbox import general
from sandbox import group
from sandbox import object
from sandbox import prefab
from sandbox import asset

import tests_common

# Flow of this testing scripts is :
# - setup -> fills the objects_to_test and assets_to_test lists
# - objects on objects_to_test are serialized to archives
# - the CRC of all the asset files in the assets_to_test list is computed,
# the asset files in xml format are serialized to archives
# - execute -> executes the functions that need testing
# - objects on objects_to_test are serialized to archives again
# - asset files CRCs and xml serialization is performed again
# - the archives serialized after setup and execute are compared, if there are discrepancies
# the test fails
# - the CRCs computed after setup and execute are compared, if there are discrepancies
# the test fails. In case the asset files are in xml format and the CRC fails XML comparison is also performed
# - the cleanup function is called so that the objects and assets created for the test can be removed

#This script creates a prefab, a group and then test the attach/detach of the group to the prefab

sphere_asset_file = "objects/default/primitive_sphere.cgf"
prefab_item_name = "itemToTest"
group_name = "test-group"
prefab_object_name = "test-prefab"

# Prepare the test, create all necessary objects and asset and add them to the lists of items (assets/objects) to test
# after this function ends all the items will be saved for comparisons
def setup():
#create the prefab hierarchy
prefab_creation_results = tests_common.create_prefab(prefab_item_name, prefab_object_name, sphere_asset_file, "Sphere", 2, 2, 0, 0, 0)

#add all the objects and assets in this hierarchy to the test lists
tests_common.list_concat(objects_to_test, prefab_creation_results.created_objects)
tests_common.list_concat(assets_to_test, prefab_creation_results.created_assets)

#create a group with 3 children
group_creation_results = tests_common.create_group(group_name, sphere_asset_file, 3, 0, 0, 0)
tests_common.list_concat(objects_to_test, group_creation_results.created_objects)

general.save()

# Execute the test, after this function ends the items in the lists (assets/objects) will be saved
# one more time and then compared to the items serialized just after the end of the setup
# function
def executeTest():
tests_common.attach_to_hierarchy(objects_to_test[0], group_name, 0)
general.undo()
general.save()

# This should destroy every object and asset created for this test
def cleanup():
#delete the group
object.delete(group_name)
#delete all the assets created for the test (and the prefab objects instantiated from the item asset)
for asset_guid in assets_to_test:
asset.delete(asset_guid.to_string())
general.save()
  • No labels