Import/Export development documentation

# AttributeCleaner
# Removes all `_ids` and other prohibited keys
    class AttributeCleaner
      ALLOWED_REFERENCES = RelationFactory::PROJECT_REFERENCES + RelationFactory::USER_REFERENCES + ['group_id']

      def clean
        @relation_hash.reject do |key, _value|
          prohibited_key?(key) || !@relation_class.attribute_method?(key) || excluded_key?(key)
        end.except('id')
      end

      ...

# AttributeConfigurationSpec
<<-MSG
  It looks like #{relation_class}, which is exported using the project Import/Export, has new attributes:

  Please add the attribute(s) to SAFE_MODEL_ATTRIBUTES if they can be exported.

  Please denylist the attribute(s) in IMPORT_EXPORT_CONFIG by adding it to its corresponding
  model in the +excluded_attributes+ section.

  SAFE_MODEL_ATTRIBUTES: #{File.expand_path(safe_attributes_file)}
  IMPORT_EXPORT_CONFIG: #{Gitlab::ImportExport.config_file}
MSG

# ModelConfigurationSpec
<<-MSG
  New model(s) <#{new_models.join(',')}> have been added, related to #{parent_model_name}, which is exported by
  the Import/Export feature.

  If you think this model should be included in the export, please add it to `#{Gitlab::ImportExport.config_file}`.

  Definitely add it to `#{File.expand_path(ce_models_yml)}`
  to signal that you've handled this error and to prevent it from showing up in the future.
MSG

# ExportFileSpec
<<-MSG
  Found a new sensitive word <#{key_found}>, which is part of the hash #{parent.inspect}
  If you think this information shouldn't get exported, please exclude the model or attribute in
  IMPORT_EXPORT_CONFIG.

  Otherwise, please add the exception to +safe_list+ in CURRENT_SPEC using #{sensitive_word} as the
  key and the correspondent hash or model as the value.

  Also, if the attribute is a generated unique token, please add it to RelationFactory::TOKEN_RESET_MODELS
  if it needs to be reset (to prevent duplicate column problems while importing to the same instance).

  IMPORT_EXPORT_CONFIG: #{Gitlab::ImportExport.config_file}
  CURRENT_SPEC: #{__FILE__}
MSG

# ImportExport
module Gitlab
  module ImportExport
    extend self

    # For every version update, the history in import_export.md has to be kept up to date.
    VERSION = '0.2.4'

bundle exec rake gitlab:import_export:bump_version

project_tree:
  - labels:
    - :priorities
  - milestones:
    - events:
      - :push_event_payload
  - issues:
    - events:
    # ...

included_attributes:
  user:
    - :id
    - :public_email
  # ...

excluded_attributes:
  project:
    - :name
    - :path
    - ...

# Methods
methods:
  labels:
    - :type
  label:
    - :type

# AttributeConfigurationSpec
<<-MSG
  It looks like #{relation_class}, which is exported using the project Import/Export, has new attributes:

  Please add the attribute(s) to SAFE_MODEL_ATTRIBUTES if they can be exported.

  Please denylist the attribute(s) in IMPORT_EXPORT_CONFIG by adding it to its corresponding
  model in the +excluded_attributes+ section.

  SAFE_MODEL_ATTRIBUTES: #{File.expand_path(safe_attributes_file)}
  IMPORT_EXPORT_CONFIG: #{Gitlab::ImportExport.config_file}
MSG
```0

### Conditional export

When associated resources are from outside the project, you might need to
validate that a user who is exporting the project or group can access these
associations. `include_if_exportable` accepts an array of associations for a
resource. During export, the `exportable_association?` method on the resource
is called with the association's name and user to validate if associated
resource can be included in the export.

For example:

```ruby
# AttributeConfigurationSpec
<<-MSG
  It looks like #{relation_class}, which is exported using the project Import/Export, has new attributes:

  Please add the attribute(s) to SAFE_MODEL_ATTRIBUTES if they can be exported.

  Please denylist the attribute(s) in IMPORT_EXPORT_CONFIG by adding it to its corresponding
  model in the +excluded_attributes+ section.

  SAFE_MODEL_ATTRIBUTES: #{File.expand_path(safe_attributes_file)}
  IMPORT_EXPORT_CONFIG: #{Gitlab::ImportExport.config_file}
MSG
```1

This definition:

1. Calls the issue's `exportable_association?(:epic_issue, current_user: current_user)` method.
1. If the method returns true, includes the issue's `epic_issue` association for the issue.

### Import

The import job status moves from `none` to `finished` or `failed` into different states:

_import\_status_: none -> scheduled -> started -> finished/failed

While the status is `started` the `Importer` code processes each step required for the import.

```ruby
# AttributeConfigurationSpec
<<-MSG
  It looks like #{relation_class}, which is exported using the project Import/Export, has new attributes:

  Please add the attribute(s) to SAFE_MODEL_ATTRIBUTES if they can be exported.

  Please denylist the attribute(s) in IMPORT_EXPORT_CONFIG by adding it to its corresponding
  model in the +excluded_attributes+ section.

  SAFE_MODEL_ATTRIBUTES: #{File.expand_path(safe_attributes_file)}
  IMPORT_EXPORT_CONFIG: #{Gitlab::ImportExport.config_file}
MSG
```2

The export service, is similar to the `Importer`, restoring data instead of saving it.

### Export

```ruby
# AttributeConfigurationSpec
<<-MSG
  It looks like #{relation_class}, which is exported using the project Import/Export, has new attributes:

  Please add the attribute(s) to SAFE_MODEL_ATTRIBUTES if they can be exported.

  Please denylist the attribute(s) in IMPORT_EXPORT_CONFIG by adding it to its corresponding
  model in the +excluded_attributes+ section.

  SAFE_MODEL_ATTRIBUTES: #{File.expand_path(safe_attributes_file)}
  IMPORT_EXPORT_CONFIG: #{Gitlab::ImportExport.config_file}
MSG
```3

## Test fixtures

Fixtures used in Import/Export specs live in `spec/fixtures/lib/gitlab/import_export`. There are both Project and Group fixtures.

There are two versions of each of these fixtures:

- A human readable single JSON file with all objects, called either `project.json` or `group.json`.
- A folder named `tree`, containing a tree of files in `ndjson` format. **Do not edit files under this folder manually unless strictly necessary.**

The tools to generate the NDJSON tree from the human-readable JSON files live in the [`gitlab-org/cloud-connector-team/team-tools`](https://gitlab.com/gitlab-org/cloud-connector-team/team-tools/-/tree/master/import-export) project.

### Project

**Use `legacy-project-json-to-ndjson.sh` to generate the NDJSON tree.**

The NDJSON tree looks like:

```ruby
# AttributeConfigurationSpec
<<-MSG
  It looks like #{relation_class}, which is exported using the project Import/Export, has new attributes:

  Please add the attribute(s) to SAFE_MODEL_ATTRIBUTES if they can be exported.

  Please denylist the attribute(s) in IMPORT_EXPORT_CONFIG by adding it to its corresponding
  model in the +excluded_attributes+ section.

  SAFE_MODEL_ATTRIBUTES: #{File.expand_path(safe_attributes_file)}
  IMPORT_EXPORT_CONFIG: #{Gitlab::ImportExport.config_file}
MSG
```4

### Group

**Use `legacy-group-json-to-ndjson.rb` to generate the NDJSON tree.**

The NDJSON tree looks like this:

```ruby
# AttributeConfigurationSpec
<<-MSG
  It looks like #{relation_class}, which is exported using the project Import/Export, has new attributes:

  Please add the attribute(s) to SAFE_MODEL_ATTRIBUTES if they can be exported.

  Please denylist the attribute(s) in IMPORT_EXPORT_CONFIG by adding it to its corresponding
  model in the +excluded_attributes+ section.

  SAFE_MODEL_ATTRIBUTES: #{File.expand_path(safe_attributes_file)}
  IMPORT_EXPORT_CONFIG: #{Gitlab::ImportExport.config_file}
MSG
```5

WARNING:
When updating these fixtures, ensure you update both `json` files and `tree` folder, as the tests apply to both.