Docs Home Pulumi IaC Pulumi ESC Pulumi Insights Pulumi Cloud Packages Tutorials
  1. Packages
  2. Mongodbatlas Provider
  3. API Docs
  4. DataLakePipeline
MongoDB Atlas v3.30.0 published on Friday, Mar 21, 2025 by Pulumi
pulumi/pulumi-mongodbatlas

mongodbatlas.DataLakePipeline

Explore with Pulumi AI

MongoDB Atlas v3.30.0 published on Friday, Mar 21, 2025 by Pulumi
pulumi/pulumi-mongodbatlas

WARNING: Data Lake is deprecated. To learn more, see https://dochub.mongodb.org/core/data-lake-deprecation

mongodbatlas.DataLakePipeline provides a Data Lake Pipeline resource.

NOTE: Groups and projects are synonymous terms. You may find group_id in the official documentation.

Example Usage

S

Coming soon!

Coming soon!

Coming soon!

Coming soon!

package generated_program;

import com.pulumi.Context;
import com.pulumi.Pulumi;
import com.pulumi.core.Output;
import com.pulumi.mongodbatlas.Project;
import com.pulumi.mongodbatlas.ProjectArgs;
import com.pulumi.mongodbatlas.AdvancedCluster;
import com.pulumi.mongodbatlas.AdvancedClusterArgs;
import com.pulumi.mongodbatlas.inputs.AdvancedClusterReplicationSpecArgs;
import com.pulumi.mongodbatlas.DataLakePipeline;
import com.pulumi.mongodbatlas.DataLakePipelineArgs;
import com.pulumi.mongodbatlas.inputs.DataLakePipelineSinkArgs;
import com.pulumi.mongodbatlas.inputs.DataLakePipelineSourceArgs;
import com.pulumi.mongodbatlas.inputs.DataLakePipelineTransformationArgs;
import java.util.List;
import java.util.ArrayList;
import java.util.Map;
import java.io.File;
import java.nio.file.Files;
import java.nio.file.Paths;

public class App {
    public static void main(String[] args) {
        Pulumi.run(App::stack);
    }

    public static void stack(Context ctx) {
        var projectTest = new Project("projectTest", ProjectArgs.builder()
            .name("NAME OF THE PROJECT")
            .orgId("ORGANIZATION ID")
            .build());

        var automatedBackupTest = new AdvancedCluster("automatedBackupTest", AdvancedClusterArgs.builder()
            .projectId(projectId)
            .name("automated-backup-test")
            .clusterType("REPLICASET")
            .backupEnabled(true)
            .replicationSpecs(AdvancedClusterReplicationSpecArgs.builder()
                .regionConfigs(AdvancedClusterReplicationSpecRegionConfigArgs.builder()
                    .priority(7)
                    .providerName("GCP")
                    .regionName("US_EAST_4")
                    .electableSpecs(AdvancedClusterReplicationSpecRegionConfigElectableSpecsArgs.builder()
                        .instanceSize("M10")
                        .nodeCount(3)
                        .build())
                    .build())
                .build())
            .build());

        var pipeline = new DataLakePipeline("pipeline", DataLakePipelineArgs.builder()
            .projectId(projectTest.projectId())
            .name("DataLakePipelineName")
            .sink(DataLakePipelineSinkArgs.builder()
                .type("DLS")
                .partitionFields(DataLakePipelineSinkPartitionFieldArgs.builder()
                    .name("access")
                    .order(0)
                    .build())
                .build())
            .source(DataLakePipelineSourceArgs.builder()
                .type("ON_DEMAND_CPS")
                .clusterName(automatedBackupTest.name())
                .databaseName("sample_airbnb")
                .collectionName("listingsAndReviews")
                .build())
            .transformations(            
                DataLakePipelineTransformationArgs.builder()
                    .field("test")
                    .type("EXCLUDE")
                    .build(),
                DataLakePipelineTransformationArgs.builder()
                    .field("test22")
                    .type("EXCLUDE")
                    .build())
            .build());

    }
}
Copy
resources:
  projectTest:
    type: mongodbatlas:Project
    properties:
      name: NAME OF THE PROJECT
      orgId: ORGANIZATION ID
  automatedBackupTest:
    type: mongodbatlas:AdvancedCluster
    name: automated_backup_test
    properties:
      projectId: ${projectId}
      name: automated-backup-test
      clusterType: REPLICASET
      backupEnabled: true # enable cloud backup snapshots
      replicationSpecs:
        - regionConfigs:
            - priority: 7
              providerName: GCP
              regionName: US_EAST_4
              electableSpecs:
                instanceSize: M10
                nodeCount: 3
  pipeline:
    type: mongodbatlas:DataLakePipeline
    properties:
      projectId: ${projectTest.projectId}
      name: DataLakePipelineName
      sink:
        type: DLS
        partitionFields:
          - name: access
            order: 0
      source:
        type: ON_DEMAND_CPS
        clusterName: ${automatedBackupTest.name}
        databaseName: sample_airbnb
        collectionName: listingsAndReviews
      transformations:
        - field: test
          type: EXCLUDE
        - field: test22
          type: EXCLUDE
Copy

Create DataLakePipeline Resource

Resources are created with functions called constructors. To learn more about declaring and configuring resources, see Resources.

Constructor syntax

new DataLakePipeline(name: string, args: DataLakePipelineArgs, opts?: CustomResourceOptions);
@overload
def DataLakePipeline(resource_name: str,
                     args: DataLakePipelineArgs,
                     opts: Optional[ResourceOptions] = None)

@overload
def DataLakePipeline(resource_name: str,
                     opts: Optional[ResourceOptions] = None,
                     project_id: Optional[str] = None,
                     name: Optional[str] = None,
                     sink: Optional[DataLakePipelineSinkArgs] = None,
                     source: Optional[DataLakePipelineSourceArgs] = None,
                     transformations: Optional[Sequence[DataLakePipelineTransformationArgs]] = None)
func NewDataLakePipeline(ctx *Context, name string, args DataLakePipelineArgs, opts ...ResourceOption) (*DataLakePipeline, error)
public DataLakePipeline(string name, DataLakePipelineArgs args, CustomResourceOptions? opts = null)
public DataLakePipeline(String name, DataLakePipelineArgs args)
public DataLakePipeline(String name, DataLakePipelineArgs args, CustomResourceOptions options)

type: mongodbatlas:DataLakePipeline
properties: # The arguments to resource properties.
options: # Bag of options to control resource's behavior.

Parameters

name This property is required. string
The unique name of the resource.
args This property is required. DataLakePipelineArgs
The arguments to resource properties.
opts CustomResourceOptions
Bag of options to control resource's behavior.
resource_name This property is required. str
The unique name of the resource.
args This property is required. DataLakePipelineArgs
The arguments to resource properties.
opts ResourceOptions
Bag of options to control resource's behavior.
ctx Context
Context object for the current deployment.
name This property is required. string
The unique name of the resource.
args This property is required. DataLakePipelineArgs
The arguments to resource properties.
opts ResourceOption
Bag of options to control resource's behavior.
name This property is required. string
The unique name of the resource.
args This property is required. DataLakePipelineArgs
The arguments to resource properties.
opts CustomResourceOptions
Bag of options to control resource's behavior.
name This property is required. String
The unique name of the resource.
args This property is required. DataLakePipelineArgs
The arguments to resource properties.
options CustomResourceOptions
Bag of options to control resource's behavior.

Constructor example

The following reference example uses placeholder values for all input properties.

var dataLakePipelineResource = new Mongodbatlas.DataLakePipeline("dataLakePipelineResource", new()
{
    ProjectId = "string",
    Name = "string",
    Sink = new Mongodbatlas.Inputs.DataLakePipelineSinkArgs
    {
        PartitionFields = new[]
        {
            new Mongodbatlas.Inputs.DataLakePipelineSinkPartitionFieldArgs
            {
                FieldName = "string",
                Order = 0,
            },
        },
        Provider = "string",
        Region = "string",
        Type = "string",
    },
    Source = new Mongodbatlas.Inputs.DataLakePipelineSourceArgs
    {
        ClusterName = "string",
        CollectionName = "string",
        DatabaseName = "string",
        PolicyItemId = "string",
        ProjectId = "string",
        Type = "string",
    },
    Transformations = new[]
    {
        new Mongodbatlas.Inputs.DataLakePipelineTransformationArgs
        {
            Field = "string",
            Type = "string",
        },
    },
});
Copy
example, err := mongodbatlas.NewDataLakePipeline(ctx, "dataLakePipelineResource", &mongodbatlas.DataLakePipelineArgs{
	ProjectId: pulumi.String("string"),
	Name:      pulumi.String("string"),
	Sink: &mongodbatlas.DataLakePipelineSinkArgs{
		PartitionFields: mongodbatlas.DataLakePipelineSinkPartitionFieldArray{
			&mongodbatlas.DataLakePipelineSinkPartitionFieldArgs{
				FieldName: pulumi.String("string"),
				Order:     pulumi.Int(0),
			},
		},
		Provider: pulumi.String("string"),
		Region:   pulumi.String("string"),
		Type:     pulumi.String("string"),
	},
	Source: &mongodbatlas.DataLakePipelineSourceArgs{
		ClusterName:    pulumi.String("string"),
		CollectionName: pulumi.String("string"),
		DatabaseName:   pulumi.String("string"),
		PolicyItemId:   pulumi.String("string"),
		ProjectId:      pulumi.String("string"),
		Type:           pulumi.String("string"),
	},
	Transformations: mongodbatlas.DataLakePipelineTransformationArray{
		&mongodbatlas.DataLakePipelineTransformationArgs{
			Field: pulumi.String("string"),
			Type:  pulumi.String("string"),
		},
	},
})
Copy
var dataLakePipelineResource = new DataLakePipeline("dataLakePipelineResource", DataLakePipelineArgs.builder()
    .projectId("string")
    .name("string")
    .sink(DataLakePipelineSinkArgs.builder()
        .partitionFields(DataLakePipelineSinkPartitionFieldArgs.builder()
            .fieldName("string")
            .order(0)
            .build())
        .provider("string")
        .region("string")
        .type("string")
        .build())
    .source(DataLakePipelineSourceArgs.builder()
        .clusterName("string")
        .collectionName("string")
        .databaseName("string")
        .policyItemId("string")
        .projectId("string")
        .type("string")
        .build())
    .transformations(DataLakePipelineTransformationArgs.builder()
        .field("string")
        .type("string")
        .build())
    .build());
Copy
data_lake_pipeline_resource = mongodbatlas.DataLakePipeline("dataLakePipelineResource",
    project_id="string",
    name="string",
    sink={
        "partition_fields": [{
            "field_name": "string",
            "order": 0,
        }],
        "provider": "string",
        "region": "string",
        "type": "string",
    },
    source={
        "cluster_name": "string",
        "collection_name": "string",
        "database_name": "string",
        "policy_item_id": "string",
        "project_id": "string",
        "type": "string",
    },
    transformations=[{
        "field": "string",
        "type": "string",
    }])
Copy
const dataLakePipelineResource = new mongodbatlas.DataLakePipeline("dataLakePipelineResource", {
    projectId: "string",
    name: "string",
    sink: {
        partitionFields: [{
            fieldName: "string",
            order: 0,
        }],
        provider: "string",
        region: "string",
        type: "string",
    },
    source: {
        clusterName: "string",
        collectionName: "string",
        databaseName: "string",
        policyItemId: "string",
        projectId: "string",
        type: "string",
    },
    transformations: [{
        field: "string",
        type: "string",
    }],
});
Copy
type: mongodbatlas:DataLakePipeline
properties:
    name: string
    projectId: string
    sink:
        partitionFields:
            - fieldName: string
              order: 0
        provider: string
        region: string
        type: string
    source:
        clusterName: string
        collectionName: string
        databaseName: string
        policyItemId: string
        projectId: string
        type: string
    transformations:
        - field: string
          type: string
Copy

DataLakePipeline Resource Properties

To learn more about resource properties and how to use them, see Inputs and Outputs in the Architecture and Concepts docs.

Inputs

In Python, inputs that are objects can be passed either as argument classes or as dictionary literals.

The DataLakePipeline resource accepts the following input properties:

ProjectId This property is required. string
The unique ID for the project to create a data lake pipeline.
Name string
Name of the Atlas Data Lake Pipeline.
Sink DataLakePipelineSink
Source DataLakePipelineSource
Transformations List<DataLakePipelineTransformation>
Fields to be excluded for this Data Lake Pipeline.

  • transformations.#.field - Key in the document.
  • transformations.#.type - Type of transformation applied during the export of the namespace in a Data Lake Pipeline.
ProjectId This property is required. string
The unique ID for the project to create a data lake pipeline.
Name string
Name of the Atlas Data Lake Pipeline.
Sink DataLakePipelineSinkArgs
Source DataLakePipelineSourceArgs
Transformations []DataLakePipelineTransformationArgs
Fields to be excluded for this Data Lake Pipeline.

  • transformations.#.field - Key in the document.
  • transformations.#.type - Type of transformation applied during the export of the namespace in a Data Lake Pipeline.
projectId This property is required. String
The unique ID for the project to create a data lake pipeline.
name String
Name of the Atlas Data Lake Pipeline.
sink DataLakePipelineSink
source DataLakePipelineSource
transformations List<DataLakePipelineTransformation>
Fields to be excluded for this Data Lake Pipeline.

  • transformations.#.field - Key in the document.
  • transformations.#.type - Type of transformation applied during the export of the namespace in a Data Lake Pipeline.
projectId This property is required. string
The unique ID for the project to create a data lake pipeline.
name string
Name of the Atlas Data Lake Pipeline.
sink DataLakePipelineSink
source DataLakePipelineSource
transformations DataLakePipelineTransformation[]
Fields to be excluded for this Data Lake Pipeline.

  • transformations.#.field - Key in the document.
  • transformations.#.type - Type of transformation applied during the export of the namespace in a Data Lake Pipeline.
project_id This property is required. str
The unique ID for the project to create a data lake pipeline.
name str
Name of the Atlas Data Lake Pipeline.
sink DataLakePipelineSinkArgs
source DataLakePipelineSourceArgs
transformations Sequence[DataLakePipelineTransformationArgs]
Fields to be excluded for this Data Lake Pipeline.

  • transformations.#.field - Key in the document.
  • transformations.#.type - Type of transformation applied during the export of the namespace in a Data Lake Pipeline.
projectId This property is required. String
The unique ID for the project to create a data lake pipeline.
name String
Name of the Atlas Data Lake Pipeline.
sink Property Map
source Property Map
transformations List<Property Map>
Fields to be excluded for this Data Lake Pipeline.

  • transformations.#.field - Key in the document.
  • transformations.#.type - Type of transformation applied during the export of the namespace in a Data Lake Pipeline.

Outputs

All input properties are implicitly available as output properties. Additionally, the DataLakePipeline resource produces the following output properties:

CreatedDate string
Timestamp that indicates when the Data Lake Pipeline was created.
Id string
The provider-assigned unique ID for this managed resource.
IngestionSchedules List<DataLakePipelineIngestionSchedule>
List of backup schedule policy items that you can use as a Data Lake Pipeline source.

  • ingestion_schedules.#.id - Unique 24-hexadecimal digit string that identifies this backup policy item.
  • ingestion_schedules.#.frequency_type - Human-readable label that identifies the frequency type associated with the backup policy.
  • ingestion_schedules.#.frequency_interval - Number that indicates the frequency interval for a set of snapshots.
  • ingestion_schedules.#.retention_unit - Unit of time in which MongoDB Atlas measures snapshot retention.
  • ingestion_schedules.#.retention_value - Duration in days, weeks, or months that MongoDB Atlas retains the snapshot.
LastUpdatedDate string
Timestamp that indicates the last time that the Data Lake Pipeline was updated.
Snapshots List<DataLakePipelineSnapshot>
List of backup snapshots that you can use to trigger an on demand pipeline run.

  • snapshots.#.id - Unique 24-hexadecimal digit string that identifies the snapshot.
  • snapshots.#.provider - Human-readable label that identifies the cloud provider that stores this snapshot.
  • snapshots.#.created_at - Date and time when MongoDB Atlas took the snapshot.
  • snapshots.#.expires_at - Date and time when MongoDB Atlas deletes the snapshot.
  • snapshots.#.frequency_type - Human-readable label that identifies how often this snapshot triggers.
  • snapshots.#.master_key - Unique string that identifies the Amazon Web Services (AWS) Key Management Service (KMS) Customer Master Key (CMK) used to encrypt the snapshot.
  • snapshots.#.mongod_version - Version of the MongoDB host that this snapshot backs up.
  • snapshots.#.replica_set_name - Human-readable label that identifies the replica set from which MongoDB Atlas took this snapshot.
  • snapshots.#.type - Human-readable label that categorizes the cluster as a replica set or sharded cluster.
  • snapshots.#.snapshot_type - Human-readable label that identifies when this snapshot triggers.
  • snapshots.#.status - Human-readable label that indicates the stage of the backup process for this snapshot.
  • snapshots.#.size - List of backup snapshots that you can use to trigger an on demand pipeline run.
  • snapshots.#.copy_region - List that identifies the regions to which MongoDB Atlas copies the snapshot.
  • snapshots.#.policies - List that contains unique identifiers for the policy items.
State string
State of this Data Lake Pipeline.
CreatedDate string
Timestamp that indicates when the Data Lake Pipeline was created.
Id string
The provider-assigned unique ID for this managed resource.
IngestionSchedules []DataLakePipelineIngestionSchedule
List of backup schedule policy items that you can use as a Data Lake Pipeline source.

  • ingestion_schedules.#.id - Unique 24-hexadecimal digit string that identifies this backup policy item.
  • ingestion_schedules.#.frequency_type - Human-readable label that identifies the frequency type associated with the backup policy.
  • ingestion_schedules.#.frequency_interval - Number that indicates the frequency interval for a set of snapshots.
  • ingestion_schedules.#.retention_unit - Unit of time in which MongoDB Atlas measures snapshot retention.
  • ingestion_schedules.#.retention_value - Duration in days, weeks, or months that MongoDB Atlas retains the snapshot.
LastUpdatedDate string
Timestamp that indicates the last time that the Data Lake Pipeline was updated.
Snapshots []DataLakePipelineSnapshot
List of backup snapshots that you can use to trigger an on demand pipeline run.

  • snapshots.#.id - Unique 24-hexadecimal digit string that identifies the snapshot.
  • snapshots.#.provider - Human-readable label that identifies the cloud provider that stores this snapshot.
  • snapshots.#.created_at - Date and time when MongoDB Atlas took the snapshot.
  • snapshots.#.expires_at - Date and time when MongoDB Atlas deletes the snapshot.
  • snapshots.#.frequency_type - Human-readable label that identifies how often this snapshot triggers.
  • snapshots.#.master_key - Unique string that identifies the Amazon Web Services (AWS) Key Management Service (KMS) Customer Master Key (CMK) used to encrypt the snapshot.
  • snapshots.#.mongod_version - Version of the MongoDB host that this snapshot backs up.
  • snapshots.#.replica_set_name - Human-readable label that identifies the replica set from which MongoDB Atlas took this snapshot.
  • snapshots.#.type - Human-readable label that categorizes the cluster as a replica set or sharded cluster.
  • snapshots.#.snapshot_type - Human-readable label that identifies when this snapshot triggers.
  • snapshots.#.status - Human-readable label that indicates the stage of the backup process for this snapshot.
  • snapshots.#.size - List of backup snapshots that you can use to trigger an on demand pipeline run.
  • snapshots.#.copy_region - List that identifies the regions to which MongoDB Atlas copies the snapshot.
  • snapshots.#.policies - List that contains unique identifiers for the policy items.
State string
State of this Data Lake Pipeline.
createdDate String
Timestamp that indicates when the Data Lake Pipeline was created.
id String
The provider-assigned unique ID for this managed resource.
ingestionSchedules List<DataLakePipelineIngestionSchedule>
List of backup schedule policy items that you can use as a Data Lake Pipeline source.

  • ingestion_schedules.#.id - Unique 24-hexadecimal digit string that identifies this backup policy item.
  • ingestion_schedules.#.frequency_type - Human-readable label that identifies the frequency type associated with the backup policy.
  • ingestion_schedules.#.frequency_interval - Number that indicates the frequency interval for a set of snapshots.
  • ingestion_schedules.#.retention_unit - Unit of time in which MongoDB Atlas measures snapshot retention.
  • ingestion_schedules.#.retention_value - Duration in days, weeks, or months that MongoDB Atlas retains the snapshot.
lastUpdatedDate String
Timestamp that indicates the last time that the Data Lake Pipeline was updated.
snapshots List<DataLakePipelineSnapshot>
List of backup snapshots that you can use to trigger an on demand pipeline run.

  • snapshots.#.id - Unique 24-hexadecimal digit string that identifies the snapshot.
  • snapshots.#.provider - Human-readable label that identifies the cloud provider that stores this snapshot.
  • snapshots.#.created_at - Date and time when MongoDB Atlas took the snapshot.
  • snapshots.#.expires_at - Date and time when MongoDB Atlas deletes the snapshot.
  • snapshots.#.frequency_type - Human-readable label that identifies how often this snapshot triggers.
  • snapshots.#.master_key - Unique string that identifies the Amazon Web Services (AWS) Key Management Service (KMS) Customer Master Key (CMK) used to encrypt the snapshot.
  • snapshots.#.mongod_version - Version of the MongoDB host that this snapshot backs up.
  • snapshots.#.replica_set_name - Human-readable label that identifies the replica set from which MongoDB Atlas took this snapshot.
  • snapshots.#.type - Human-readable label that categorizes the cluster as a replica set or sharded cluster.
  • snapshots.#.snapshot_type - Human-readable label that identifies when this snapshot triggers.
  • snapshots.#.status - Human-readable label that indicates the stage of the backup process for this snapshot.
  • snapshots.#.size - List of backup snapshots that you can use to trigger an on demand pipeline run.
  • snapshots.#.copy_region - List that identifies the regions to which MongoDB Atlas copies the snapshot.
  • snapshots.#.policies - List that contains unique identifiers for the policy items.
state String
State of this Data Lake Pipeline.
createdDate string
Timestamp that indicates when the Data Lake Pipeline was created.
id string
The provider-assigned unique ID for this managed resource.
ingestionSchedules DataLakePipelineIngestionSchedule[]
List of backup schedule policy items that you can use as a Data Lake Pipeline source.

  • ingestion_schedules.#.id - Unique 24-hexadecimal digit string that identifies this backup policy item.
  • ingestion_schedules.#.frequency_type - Human-readable label that identifies the frequency type associated with the backup policy.
  • ingestion_schedules.#.frequency_interval - Number that indicates the frequency interval for a set of snapshots.
  • ingestion_schedules.#.retention_unit - Unit of time in which MongoDB Atlas measures snapshot retention.
  • ingestion_schedules.#.retention_value - Duration in days, weeks, or months that MongoDB Atlas retains the snapshot.
lastUpdatedDate string
Timestamp that indicates the last time that the Data Lake Pipeline was updated.
snapshots DataLakePipelineSnapshot[]
List of backup snapshots that you can use to trigger an on demand pipeline run.

  • snapshots.#.id - Unique 24-hexadecimal digit string that identifies the snapshot.
  • snapshots.#.provider - Human-readable label that identifies the cloud provider that stores this snapshot.
  • snapshots.#.created_at - Date and time when MongoDB Atlas took the snapshot.
  • snapshots.#.expires_at - Date and time when MongoDB Atlas deletes the snapshot.
  • snapshots.#.frequency_type - Human-readable label that identifies how often this snapshot triggers.
  • snapshots.#.master_key - Unique string that identifies the Amazon Web Services (AWS) Key Management Service (KMS) Customer Master Key (CMK) used to encrypt the snapshot.
  • snapshots.#.mongod_version - Version of the MongoDB host that this snapshot backs up.
  • snapshots.#.replica_set_name - Human-readable label that identifies the replica set from which MongoDB Atlas took this snapshot.
  • snapshots.#.type - Human-readable label that categorizes the cluster as a replica set or sharded cluster.
  • snapshots.#.snapshot_type - Human-readable label that identifies when this snapshot triggers.
  • snapshots.#.status - Human-readable label that indicates the stage of the backup process for this snapshot.
  • snapshots.#.size - List of backup snapshots that you can use to trigger an on demand pipeline run.
  • snapshots.#.copy_region - List that identifies the regions to which MongoDB Atlas copies the snapshot.
  • snapshots.#.policies - List that contains unique identifiers for the policy items.
state string
State of this Data Lake Pipeline.
created_date str
Timestamp that indicates when the Data Lake Pipeline was created.
id str
The provider-assigned unique ID for this managed resource.
ingestion_schedules Sequence[DataLakePipelineIngestionSchedule]
List of backup schedule policy items that you can use as a Data Lake Pipeline source.

  • ingestion_schedules.#.id - Unique 24-hexadecimal digit string that identifies this backup policy item.
  • ingestion_schedules.#.frequency_type - Human-readable label that identifies the frequency type associated with the backup policy.
  • ingestion_schedules.#.frequency_interval - Number that indicates the frequency interval for a set of snapshots.
  • ingestion_schedules.#.retention_unit - Unit of time in which MongoDB Atlas measures snapshot retention.
  • ingestion_schedules.#.retention_value - Duration in days, weeks, or months that MongoDB Atlas retains the snapshot.
last_updated_date str
Timestamp that indicates the last time that the Data Lake Pipeline was updated.
snapshots Sequence[DataLakePipelineSnapshot]
List of backup snapshots that you can use to trigger an on demand pipeline run.

  • snapshots.#.id - Unique 24-hexadecimal digit string that identifies the snapshot.
  • snapshots.#.provider - Human-readable label that identifies the cloud provider that stores this snapshot.
  • snapshots.#.created_at - Date and time when MongoDB Atlas took the snapshot.
  • snapshots.#.expires_at - Date and time when MongoDB Atlas deletes the snapshot.
  • snapshots.#.frequency_type - Human-readable label that identifies how often this snapshot triggers.
  • snapshots.#.master_key - Unique string that identifies the Amazon Web Services (AWS) Key Management Service (KMS) Customer Master Key (CMK) used to encrypt the snapshot.
  • snapshots.#.mongod_version - Version of the MongoDB host that this snapshot backs up.
  • snapshots.#.replica_set_name - Human-readable label that identifies the replica set from which MongoDB Atlas took this snapshot.
  • snapshots.#.type - Human-readable label that categorizes the cluster as a replica set or sharded cluster.
  • snapshots.#.snapshot_type - Human-readable label that identifies when this snapshot triggers.
  • snapshots.#.status - Human-readable label that indicates the stage of the backup process for this snapshot.
  • snapshots.#.size - List of backup snapshots that you can use to trigger an on demand pipeline run.
  • snapshots.#.copy_region - List that identifies the regions to which MongoDB Atlas copies the snapshot.
  • snapshots.#.policies - List that contains unique identifiers for the policy items.
state str
State of this Data Lake Pipeline.
createdDate String
Timestamp that indicates when the Data Lake Pipeline was created.
id String
The provider-assigned unique ID for this managed resource.
ingestionSchedules List<Property Map>
List of backup schedule policy items that you can use as a Data Lake Pipeline source.

  • ingestion_schedules.#.id - Unique 24-hexadecimal digit string that identifies this backup policy item.
  • ingestion_schedules.#.frequency_type - Human-readable label that identifies the frequency type associated with the backup policy.
  • ingestion_schedules.#.frequency_interval - Number that indicates the frequency interval for a set of snapshots.
  • ingestion_schedules.#.retention_unit - Unit of time in which MongoDB Atlas measures snapshot retention.
  • ingestion_schedules.#.retention_value - Duration in days, weeks, or months that MongoDB Atlas retains the snapshot.
lastUpdatedDate String
Timestamp that indicates the last time that the Data Lake Pipeline was updated.
snapshots List<Property Map>
List of backup snapshots that you can use to trigger an on demand pipeline run.

  • snapshots.#.id - Unique 24-hexadecimal digit string that identifies the snapshot.
  • snapshots.#.provider - Human-readable label that identifies the cloud provider that stores this snapshot.
  • snapshots.#.created_at - Date and time when MongoDB Atlas took the snapshot.
  • snapshots.#.expires_at - Date and time when MongoDB Atlas deletes the snapshot.
  • snapshots.#.frequency_type - Human-readable label that identifies how often this snapshot triggers.
  • snapshots.#.master_key - Unique string that identifies the Amazon Web Services (AWS) Key Management Service (KMS) Customer Master Key (CMK) used to encrypt the snapshot.
  • snapshots.#.mongod_version - Version of the MongoDB host that this snapshot backs up.
  • snapshots.#.replica_set_name - Human-readable label that identifies the replica set from which MongoDB Atlas took this snapshot.
  • snapshots.#.type - Human-readable label that categorizes the cluster as a replica set or sharded cluster.
  • snapshots.#.snapshot_type - Human-readable label that identifies when this snapshot triggers.
  • snapshots.#.status - Human-readable label that indicates the stage of the backup process for this snapshot.
  • snapshots.#.size - List of backup snapshots that you can use to trigger an on demand pipeline run.
  • snapshots.#.copy_region - List that identifies the regions to which MongoDB Atlas copies the snapshot.
  • snapshots.#.policies - List that contains unique identifiers for the policy items.
state String
State of this Data Lake Pipeline.

Look up Existing DataLakePipeline Resource

Get an existing DataLakePipeline resource’s state with the given name, ID, and optional extra properties used to qualify the lookup.

public static get(name: string, id: Input<ID>, state?: DataLakePipelineState, opts?: CustomResourceOptions): DataLakePipeline
@staticmethod
def get(resource_name: str,
        id: str,
        opts: Optional[ResourceOptions] = None,
        created_date: Optional[str] = None,
        ingestion_schedules: Optional[Sequence[DataLakePipelineIngestionScheduleArgs]] = None,
        last_updated_date: Optional[str] = None,
        name: Optional[str] = None,
        project_id: Optional[str] = None,
        sink: Optional[DataLakePipelineSinkArgs] = None,
        snapshots: Optional[Sequence[DataLakePipelineSnapshotArgs]] = None,
        source: Optional[DataLakePipelineSourceArgs] = None,
        state: Optional[str] = None,
        transformations: Optional[Sequence[DataLakePipelineTransformationArgs]] = None) -> DataLakePipeline
func GetDataLakePipeline(ctx *Context, name string, id IDInput, state *DataLakePipelineState, opts ...ResourceOption) (*DataLakePipeline, error)
public static DataLakePipeline Get(string name, Input<string> id, DataLakePipelineState? state, CustomResourceOptions? opts = null)
public static DataLakePipeline get(String name, Output<String> id, DataLakePipelineState state, CustomResourceOptions options)
resources:  _:    type: mongodbatlas:DataLakePipeline    get:      id: ${id}
name This property is required.
The unique name of the resulting resource.
id This property is required.
The unique provider ID of the resource to lookup.
state
Any extra arguments used during the lookup.
opts
A bag of options that control this resource's behavior.
resource_name This property is required.
The unique name of the resulting resource.
id This property is required.
The unique provider ID of the resource to lookup.
name This property is required.
The unique name of the resulting resource.
id This property is required.
The unique provider ID of the resource to lookup.
state
Any extra arguments used during the lookup.
opts
A bag of options that control this resource's behavior.
name This property is required.
The unique name of the resulting resource.
id This property is required.
The unique provider ID of the resource to lookup.
state
Any extra arguments used during the lookup.
opts
A bag of options that control this resource's behavior.
name This property is required.
The unique name of the resulting resource.
id This property is required.
The unique provider ID of the resource to lookup.
state
Any extra arguments used during the lookup.
opts
A bag of options that control this resource's behavior.
The following state arguments are supported:
CreatedDate string
Timestamp that indicates when the Data Lake Pipeline was created.
IngestionSchedules List<DataLakePipelineIngestionSchedule>
List of backup schedule policy items that you can use as a Data Lake Pipeline source.

  • ingestion_schedules.#.id - Unique 24-hexadecimal digit string that identifies this backup policy item.
  • ingestion_schedules.#.frequency_type - Human-readable label that identifies the frequency type associated with the backup policy.
  • ingestion_schedules.#.frequency_interval - Number that indicates the frequency interval for a set of snapshots.
  • ingestion_schedules.#.retention_unit - Unit of time in which MongoDB Atlas measures snapshot retention.
  • ingestion_schedules.#.retention_value - Duration in days, weeks, or months that MongoDB Atlas retains the snapshot.
LastUpdatedDate string
Timestamp that indicates the last time that the Data Lake Pipeline was updated.
Name string
Name of the Atlas Data Lake Pipeline.
ProjectId string
The unique ID for the project to create a data lake pipeline.
Sink DataLakePipelineSink
Snapshots List<DataLakePipelineSnapshot>
List of backup snapshots that you can use to trigger an on demand pipeline run.

  • snapshots.#.id - Unique 24-hexadecimal digit string that identifies the snapshot.
  • snapshots.#.provider - Human-readable label that identifies the cloud provider that stores this snapshot.
  • snapshots.#.created_at - Date and time when MongoDB Atlas took the snapshot.
  • snapshots.#.expires_at - Date and time when MongoDB Atlas deletes the snapshot.
  • snapshots.#.frequency_type - Human-readable label that identifies how often this snapshot triggers.
  • snapshots.#.master_key - Unique string that identifies the Amazon Web Services (AWS) Key Management Service (KMS) Customer Master Key (CMK) used to encrypt the snapshot.
  • snapshots.#.mongod_version - Version of the MongoDB host that this snapshot backs up.
  • snapshots.#.replica_set_name - Human-readable label that identifies the replica set from which MongoDB Atlas took this snapshot.
  • snapshots.#.type - Human-readable label that categorizes the cluster as a replica set or sharded cluster.
  • snapshots.#.snapshot_type - Human-readable label that identifies when this snapshot triggers.
  • snapshots.#.status - Human-readable label that indicates the stage of the backup process for this snapshot.
  • snapshots.#.size - List of backup snapshots that you can use to trigger an on demand pipeline run.
  • snapshots.#.copy_region - List that identifies the regions to which MongoDB Atlas copies the snapshot.
  • snapshots.#.policies - List that contains unique identifiers for the policy items.
Source DataLakePipelineSource
State string
State of this Data Lake Pipeline.
Transformations List<DataLakePipelineTransformation>
Fields to be excluded for this Data Lake Pipeline.

  • transformations.#.field - Key in the document.
  • transformations.#.type - Type of transformation applied during the export of the namespace in a Data Lake Pipeline.
CreatedDate string
Timestamp that indicates when the Data Lake Pipeline was created.
IngestionSchedules []DataLakePipelineIngestionScheduleArgs
List of backup schedule policy items that you can use as a Data Lake Pipeline source.

  • ingestion_schedules.#.id - Unique 24-hexadecimal digit string that identifies this backup policy item.
  • ingestion_schedules.#.frequency_type - Human-readable label that identifies the frequency type associated with the backup policy.
  • ingestion_schedules.#.frequency_interval - Number that indicates the frequency interval for a set of snapshots.
  • ingestion_schedules.#.retention_unit - Unit of time in which MongoDB Atlas measures snapshot retention.
  • ingestion_schedules.#.retention_value - Duration in days, weeks, or months that MongoDB Atlas retains the snapshot.
LastUpdatedDate string
Timestamp that indicates the last time that the Data Lake Pipeline was updated.
Name string
Name of the Atlas Data Lake Pipeline.
ProjectId string
The unique ID for the project to create a data lake pipeline.
Sink DataLakePipelineSinkArgs
Snapshots []DataLakePipelineSnapshotArgs
List of backup snapshots that you can use to trigger an on demand pipeline run.

  • snapshots.#.id - Unique 24-hexadecimal digit string that identifies the snapshot.
  • snapshots.#.provider - Human-readable label that identifies the cloud provider that stores this snapshot.
  • snapshots.#.created_at - Date and time when MongoDB Atlas took the snapshot.
  • snapshots.#.expires_at - Date and time when MongoDB Atlas deletes the snapshot.
  • snapshots.#.frequency_type - Human-readable label that identifies how often this snapshot triggers.
  • snapshots.#.master_key - Unique string that identifies the Amazon Web Services (AWS) Key Management Service (KMS) Customer Master Key (CMK) used to encrypt the snapshot.
  • snapshots.#.mongod_version - Version of the MongoDB host that this snapshot backs up.
  • snapshots.#.replica_set_name - Human-readable label that identifies the replica set from which MongoDB Atlas took this snapshot.
  • snapshots.#.type - Human-readable label that categorizes the cluster as a replica set or sharded cluster.
  • snapshots.#.snapshot_type - Human-readable label that identifies when this snapshot triggers.
  • snapshots.#.status - Human-readable label that indicates the stage of the backup process for this snapshot.
  • snapshots.#.size - List of backup snapshots that you can use to trigger an on demand pipeline run.
  • snapshots.#.copy_region - List that identifies the regions to which MongoDB Atlas copies the snapshot.
  • snapshots.#.policies - List that contains unique identifiers for the policy items.
Source DataLakePipelineSourceArgs
State string
State of this Data Lake Pipeline.
Transformations []DataLakePipelineTransformationArgs
Fields to be excluded for this Data Lake Pipeline.

  • transformations.#.field - Key in the document.
  • transformations.#.type - Type of transformation applied during the export of the namespace in a Data Lake Pipeline.
createdDate String
Timestamp that indicates when the Data Lake Pipeline was created.
ingestionSchedules List<DataLakePipelineIngestionSchedule>
List of backup schedule policy items that you can use as a Data Lake Pipeline source.

  • ingestion_schedules.#.id - Unique 24-hexadecimal digit string that identifies this backup policy item.
  • ingestion_schedules.#.frequency_type - Human-readable label that identifies the frequency type associated with the backup policy.
  • ingestion_schedules.#.frequency_interval - Number that indicates the frequency interval for a set of snapshots.
  • ingestion_schedules.#.retention_unit - Unit of time in which MongoDB Atlas measures snapshot retention.
  • ingestion_schedules.#.retention_value - Duration in days, weeks, or months that MongoDB Atlas retains the snapshot.
lastUpdatedDate String
Timestamp that indicates the last time that the Data Lake Pipeline was updated.
name String
Name of the Atlas Data Lake Pipeline.
projectId String
The unique ID for the project to create a data lake pipeline.
sink DataLakePipelineSink
snapshots List<DataLakePipelineSnapshot>
List of backup snapshots that you can use to trigger an on demand pipeline run.

  • snapshots.#.id - Unique 24-hexadecimal digit string that identifies the snapshot.
  • snapshots.#.provider - Human-readable label that identifies the cloud provider that stores this snapshot.
  • snapshots.#.created_at - Date and time when MongoDB Atlas took the snapshot.
  • snapshots.#.expires_at - Date and time when MongoDB Atlas deletes the snapshot.
  • snapshots.#.frequency_type - Human-readable label that identifies how often this snapshot triggers.
  • snapshots.#.master_key - Unique string that identifies the Amazon Web Services (AWS) Key Management Service (KMS) Customer Master Key (CMK) used to encrypt the snapshot.
  • snapshots.#.mongod_version - Version of the MongoDB host that this snapshot backs up.
  • snapshots.#.replica_set_name - Human-readable label that identifies the replica set from which MongoDB Atlas took this snapshot.
  • snapshots.#.type - Human-readable label that categorizes the cluster as a replica set or sharded cluster.
  • snapshots.#.snapshot_type - Human-readable label that identifies when this snapshot triggers.
  • snapshots.#.status - Human-readable label that indicates the stage of the backup process for this snapshot.
  • snapshots.#.size - List of backup snapshots that you can use to trigger an on demand pipeline run.
  • snapshots.#.copy_region - List that identifies the regions to which MongoDB Atlas copies the snapshot.
  • snapshots.#.policies - List that contains unique identifiers for the policy items.
source DataLakePipelineSource
state String
State of this Data Lake Pipeline.
transformations List<DataLakePipelineTransformation>
Fields to be excluded for this Data Lake Pipeline.

  • transformations.#.field - Key in the document.
  • transformations.#.type - Type of transformation applied during the export of the namespace in a Data Lake Pipeline.
createdDate string
Timestamp that indicates when the Data Lake Pipeline was created.
ingestionSchedules DataLakePipelineIngestionSchedule[]
List of backup schedule policy items that you can use as a Data Lake Pipeline source.

  • ingestion_schedules.#.id - Unique 24-hexadecimal digit string that identifies this backup policy item.
  • ingestion_schedules.#.frequency_type - Human-readable label that identifies the frequency type associated with the backup policy.
  • ingestion_schedules.#.frequency_interval - Number that indicates the frequency interval for a set of snapshots.
  • ingestion_schedules.#.retention_unit - Unit of time in which MongoDB Atlas measures snapshot retention.
  • ingestion_schedules.#.retention_value - Duration in days, weeks, or months that MongoDB Atlas retains the snapshot.
lastUpdatedDate string
Timestamp that indicates the last time that the Data Lake Pipeline was updated.
name string
Name of the Atlas Data Lake Pipeline.
projectId string
The unique ID for the project to create a data lake pipeline.
sink DataLakePipelineSink
snapshots DataLakePipelineSnapshot[]
List of backup snapshots that you can use to trigger an on demand pipeline run.

  • snapshots.#.id - Unique 24-hexadecimal digit string that identifies the snapshot.
  • snapshots.#.provider - Human-readable label that identifies the cloud provider that stores this snapshot.
  • snapshots.#.created_at - Date and time when MongoDB Atlas took the snapshot.
  • snapshots.#.expires_at - Date and time when MongoDB Atlas deletes the snapshot.
  • snapshots.#.frequency_type - Human-readable label that identifies how often this snapshot triggers.
  • snapshots.#.master_key - Unique string that identifies the Amazon Web Services (AWS) Key Management Service (KMS) Customer Master Key (CMK) used to encrypt the snapshot.
  • snapshots.#.mongod_version - Version of the MongoDB host that this snapshot backs up.
  • snapshots.#.replica_set_name - Human-readable label that identifies the replica set from which MongoDB Atlas took this snapshot.
  • snapshots.#.type - Human-readable label that categorizes the cluster as a replica set or sharded cluster.
  • snapshots.#.snapshot_type - Human-readable label that identifies when this snapshot triggers.
  • snapshots.#.status - Human-readable label that indicates the stage of the backup process for this snapshot.
  • snapshots.#.size - List of backup snapshots that you can use to trigger an on demand pipeline run.
  • snapshots.#.copy_region - List that identifies the regions to which MongoDB Atlas copies the snapshot.
  • snapshots.#.policies - List that contains unique identifiers for the policy items.
source DataLakePipelineSource
state string
State of this Data Lake Pipeline.
transformations DataLakePipelineTransformation[]
Fields to be excluded for this Data Lake Pipeline.

  • transformations.#.field - Key in the document.
  • transformations.#.type - Type of transformation applied during the export of the namespace in a Data Lake Pipeline.
created_date str
Timestamp that indicates when the Data Lake Pipeline was created.
ingestion_schedules Sequence[DataLakePipelineIngestionScheduleArgs]
List of backup schedule policy items that you can use as a Data Lake Pipeline source.

  • ingestion_schedules.#.id - Unique 24-hexadecimal digit string that identifies this backup policy item.
  • ingestion_schedules.#.frequency_type - Human-readable label that identifies the frequency type associated with the backup policy.
  • ingestion_schedules.#.frequency_interval - Number that indicates the frequency interval for a set of snapshots.
  • ingestion_schedules.#.retention_unit - Unit of time in which MongoDB Atlas measures snapshot retention.
  • ingestion_schedules.#.retention_value - Duration in days, weeks, or months that MongoDB Atlas retains the snapshot.
last_updated_date str
Timestamp that indicates the last time that the Data Lake Pipeline was updated.
name str
Name of the Atlas Data Lake Pipeline.
project_id str
The unique ID for the project to create a data lake pipeline.
sink DataLakePipelineSinkArgs
snapshots Sequence[DataLakePipelineSnapshotArgs]
List of backup snapshots that you can use to trigger an on demand pipeline run.

  • snapshots.#.id - Unique 24-hexadecimal digit string that identifies the snapshot.
  • snapshots.#.provider - Human-readable label that identifies the cloud provider that stores this snapshot.
  • snapshots.#.created_at - Date and time when MongoDB Atlas took the snapshot.
  • snapshots.#.expires_at - Date and time when MongoDB Atlas deletes the snapshot.
  • snapshots.#.frequency_type - Human-readable label that identifies how often this snapshot triggers.
  • snapshots.#.master_key - Unique string that identifies the Amazon Web Services (AWS) Key Management Service (KMS) Customer Master Key (CMK) used to encrypt the snapshot.
  • snapshots.#.mongod_version - Version of the MongoDB host that this snapshot backs up.
  • snapshots.#.replica_set_name - Human-readable label that identifies the replica set from which MongoDB Atlas took this snapshot.
  • snapshots.#.type - Human-readable label that categorizes the cluster as a replica set or sharded cluster.
  • snapshots.#.snapshot_type - Human-readable label that identifies when this snapshot triggers.
  • snapshots.#.status - Human-readable label that indicates the stage of the backup process for this snapshot.
  • snapshots.#.size - List of backup snapshots that you can use to trigger an on demand pipeline run.
  • snapshots.#.copy_region - List that identifies the regions to which MongoDB Atlas copies the snapshot.
  • snapshots.#.policies - List that contains unique identifiers for the policy items.
source DataLakePipelineSourceArgs
state str
State of this Data Lake Pipeline.
transformations Sequence[DataLakePipelineTransformationArgs]
Fields to be excluded for this Data Lake Pipeline.

  • transformations.#.field - Key in the document.
  • transformations.#.type - Type of transformation applied during the export of the namespace in a Data Lake Pipeline.
createdDate String
Timestamp that indicates when the Data Lake Pipeline was created.
ingestionSchedules List<Property Map>
List of backup schedule policy items that you can use as a Data Lake Pipeline source.

  • ingestion_schedules.#.id - Unique 24-hexadecimal digit string that identifies this backup policy item.
  • ingestion_schedules.#.frequency_type - Human-readable label that identifies the frequency type associated with the backup policy.
  • ingestion_schedules.#.frequency_interval - Number that indicates the frequency interval for a set of snapshots.
  • ingestion_schedules.#.retention_unit - Unit of time in which MongoDB Atlas measures snapshot retention.
  • ingestion_schedules.#.retention_value - Duration in days, weeks, or months that MongoDB Atlas retains the snapshot.
lastUpdatedDate String
Timestamp that indicates the last time that the Data Lake Pipeline was updated.
name String
Name of the Atlas Data Lake Pipeline.
projectId String
The unique ID for the project to create a data lake pipeline.
sink Property Map
snapshots List<Property Map>
List of backup snapshots that you can use to trigger an on demand pipeline run.

  • snapshots.#.id - Unique 24-hexadecimal digit string that identifies the snapshot.
  • snapshots.#.provider - Human-readable label that identifies the cloud provider that stores this snapshot.
  • snapshots.#.created_at - Date and time when MongoDB Atlas took the snapshot.
  • snapshots.#.expires_at - Date and time when MongoDB Atlas deletes the snapshot.
  • snapshots.#.frequency_type - Human-readable label that identifies how often this snapshot triggers.
  • snapshots.#.master_key - Unique string that identifies the Amazon Web Services (AWS) Key Management Service (KMS) Customer Master Key (CMK) used to encrypt the snapshot.
  • snapshots.#.mongod_version - Version of the MongoDB host that this snapshot backs up.
  • snapshots.#.replica_set_name - Human-readable label that identifies the replica set from which MongoDB Atlas took this snapshot.
  • snapshots.#.type - Human-readable label that categorizes the cluster as a replica set or sharded cluster.
  • snapshots.#.snapshot_type - Human-readable label that identifies when this snapshot triggers.
  • snapshots.#.status - Human-readable label that indicates the stage of the backup process for this snapshot.
  • snapshots.#.size - List of backup snapshots that you can use to trigger an on demand pipeline run.
  • snapshots.#.copy_region - List that identifies the regions to which MongoDB Atlas copies the snapshot.
  • snapshots.#.policies - List that contains unique identifiers for the policy items.
source Property Map
state String
State of this Data Lake Pipeline.
transformations List<Property Map>
Fields to be excluded for this Data Lake Pipeline.

  • transformations.#.field - Key in the document.
  • transformations.#.type - Type of transformation applied during the export of the namespace in a Data Lake Pipeline.

Supporting Types

DataLakePipelineIngestionSchedule
, DataLakePipelineIngestionScheduleArgs

FrequencyInterval int
FrequencyType string
Id string
Unique 24-hexadecimal digit string that identifies the Data Lake Pipeline.
RetentionUnit string
RetentionValue int
FrequencyInterval int
FrequencyType string
Id string
Unique 24-hexadecimal digit string that identifies the Data Lake Pipeline.
RetentionUnit string
RetentionValue int
frequencyInterval Integer
frequencyType String
id String
Unique 24-hexadecimal digit string that identifies the Data Lake Pipeline.
retentionUnit String
retentionValue Integer
frequencyInterval number
frequencyType string
id string
Unique 24-hexadecimal digit string that identifies the Data Lake Pipeline.
retentionUnit string
retentionValue number
frequency_interval int
frequency_type str
id str
Unique 24-hexadecimal digit string that identifies the Data Lake Pipeline.
retention_unit str
retention_value int
frequencyInterval Number
frequencyType String
id String
Unique 24-hexadecimal digit string that identifies the Data Lake Pipeline.
retentionUnit String
retentionValue Number

DataLakePipelineSink
, DataLakePipelineSinkArgs

PartitionFields List<DataLakePipelineSinkPartitionField>
Ordered fields used to physically organize data in the destination.

  • partition_fields.#.field_name - Human-readable label that identifies the field name used to partition data.
  • partition_fields.#.order - Sequence in which MongoDB Atlas slices the collection data to create partitions. The resource expresses this sequence starting with zero.
Provider string
Target cloud provider for this Data Lake Pipeline.
Region string
Target cloud provider region for this Data Lake Pipeline. Supported cloud provider regions.
Type string
Type of ingestion source of this Data Lake Pipeline.
PartitionFields []DataLakePipelineSinkPartitionField
Ordered fields used to physically organize data in the destination.

  • partition_fields.#.field_name - Human-readable label that identifies the field name used to partition data.
  • partition_fields.#.order - Sequence in which MongoDB Atlas slices the collection data to create partitions. The resource expresses this sequence starting with zero.
Provider string
Target cloud provider for this Data Lake Pipeline.
Region string
Target cloud provider region for this Data Lake Pipeline. Supported cloud provider regions.
Type string
Type of ingestion source of this Data Lake Pipeline.
partitionFields List<DataLakePipelineSinkPartitionField>
Ordered fields used to physically organize data in the destination.

  • partition_fields.#.field_name - Human-readable label that identifies the field name used to partition data.
  • partition_fields.#.order - Sequence in which MongoDB Atlas slices the collection data to create partitions. The resource expresses this sequence starting with zero.
provider String
Target cloud provider for this Data Lake Pipeline.
region String
Target cloud provider region for this Data Lake Pipeline. Supported cloud provider regions.
type String
Type of ingestion source of this Data Lake Pipeline.
partitionFields DataLakePipelineSinkPartitionField[]
Ordered fields used to physically organize data in the destination.

  • partition_fields.#.field_name - Human-readable label that identifies the field name used to partition data.
  • partition_fields.#.order - Sequence in which MongoDB Atlas slices the collection data to create partitions. The resource expresses this sequence starting with zero.
provider string
Target cloud provider for this Data Lake Pipeline.
region string
Target cloud provider region for this Data Lake Pipeline. Supported cloud provider regions.
type string
Type of ingestion source of this Data Lake Pipeline.
partition_fields Sequence[DataLakePipelineSinkPartitionField]
Ordered fields used to physically organize data in the destination.

  • partition_fields.#.field_name - Human-readable label that identifies the field name used to partition data.
  • partition_fields.#.order - Sequence in which MongoDB Atlas slices the collection data to create partitions. The resource expresses this sequence starting with zero.
provider str
Target cloud provider for this Data Lake Pipeline.
region str
Target cloud provider region for this Data Lake Pipeline. Supported cloud provider regions.
type str
Type of ingestion source of this Data Lake Pipeline.
partitionFields List<Property Map>
Ordered fields used to physically organize data in the destination.

  • partition_fields.#.field_name - Human-readable label that identifies the field name used to partition data.
  • partition_fields.#.order - Sequence in which MongoDB Atlas slices the collection data to create partitions. The resource expresses this sequence starting with zero.
provider String
Target cloud provider for this Data Lake Pipeline.
region String
Target cloud provider region for this Data Lake Pipeline. Supported cloud provider regions.
type String
Type of ingestion source of this Data Lake Pipeline.

DataLakePipelineSinkPartitionField
, DataLakePipelineSinkPartitionFieldArgs

FieldName This property is required. string
Order This property is required. int
FieldName This property is required. string
Order This property is required. int
fieldName This property is required. String
order This property is required. Integer
fieldName This property is required. string
order This property is required. number
field_name This property is required. str
order This property is required. int
fieldName This property is required. String
order This property is required. Number

DataLakePipelineSnapshot
, DataLakePipelineSnapshotArgs

CopyRegion string
CreatedAt string
ExpiresAt string
FrequencyYype string
Id string
Unique 24-hexadecimal digit string that identifies the Data Lake Pipeline.
MasterKey string
MongodVersion string
Policies List<string>
Provider string
Target cloud provider for this Data Lake Pipeline.
ReplicaSetName string
Size int
SnapshotType string
Status string
Type string
Type of ingestion source of this Data Lake Pipeline.
CopyRegion string
CreatedAt string
ExpiresAt string
FrequencyYype string
Id string
Unique 24-hexadecimal digit string that identifies the Data Lake Pipeline.
MasterKey string
MongodVersion string
Policies []string
Provider string
Target cloud provider for this Data Lake Pipeline.
ReplicaSetName string
Size int
SnapshotType string
Status string
Type string
Type of ingestion source of this Data Lake Pipeline.
copyRegion String
createdAt String
expiresAt String
frequencyYype String
id String
Unique 24-hexadecimal digit string that identifies the Data Lake Pipeline.
masterKey String
mongodVersion String
policies List<String>
provider String
Target cloud provider for this Data Lake Pipeline.
replicaSetName String
size Integer
snapshotType String
status String
type String
Type of ingestion source of this Data Lake Pipeline.
copyRegion string
createdAt string
expiresAt string
frequencyYype string
id string
Unique 24-hexadecimal digit string that identifies the Data Lake Pipeline.
masterKey string
mongodVersion string
policies string[]
provider string
Target cloud provider for this Data Lake Pipeline.
replicaSetName string
size number
snapshotType string
status string
type string
Type of ingestion source of this Data Lake Pipeline.
copy_region str
created_at str
expires_at str
frequency_yype str
id str
Unique 24-hexadecimal digit string that identifies the Data Lake Pipeline.
master_key str
mongod_version str
policies Sequence[str]
provider str
Target cloud provider for this Data Lake Pipeline.
replica_set_name str
size int
snapshot_type str
status str
type str
Type of ingestion source of this Data Lake Pipeline.
copyRegion String
createdAt String
expiresAt String
frequencyYype String
id String
Unique 24-hexadecimal digit string that identifies the Data Lake Pipeline.
masterKey String
mongodVersion String
policies List<String>
provider String
Target cloud provider for this Data Lake Pipeline.
replicaSetName String
size Number
snapshotType String
status String
type String
Type of ingestion source of this Data Lake Pipeline.

DataLakePipelineSource
, DataLakePipelineSourceArgs

ClusterName string
Human-readable name that identifies the cluster.
CollectionName string
Human-readable name that identifies the collection.
DatabaseName string
Human-readable name that identifies the database.
PolicyItemId string
ProjectId string
The unique ID for the project to create a data lake pipeline.
Type string
Type of ingestion source of this Data Lake Pipeline.
ClusterName string
Human-readable name that identifies the cluster.
CollectionName string
Human-readable name that identifies the collection.
DatabaseName string
Human-readable name that identifies the database.
PolicyItemId string
ProjectId string
The unique ID for the project to create a data lake pipeline.
Type string
Type of ingestion source of this Data Lake Pipeline.
clusterName String
Human-readable name that identifies the cluster.
collectionName String
Human-readable name that identifies the collection.
databaseName String
Human-readable name that identifies the database.
policyItemId String
projectId String
The unique ID for the project to create a data lake pipeline.
type String
Type of ingestion source of this Data Lake Pipeline.
clusterName string
Human-readable name that identifies the cluster.
collectionName string
Human-readable name that identifies the collection.
databaseName string
Human-readable name that identifies the database.
policyItemId string
projectId string
The unique ID for the project to create a data lake pipeline.
type string
Type of ingestion source of this Data Lake Pipeline.
cluster_name str
Human-readable name that identifies the cluster.
collection_name str
Human-readable name that identifies the collection.
database_name str
Human-readable name that identifies the database.
policy_item_id str
project_id str
The unique ID for the project to create a data lake pipeline.
type str
Type of ingestion source of this Data Lake Pipeline.
clusterName String
Human-readable name that identifies the cluster.
collectionName String
Human-readable name that identifies the collection.
databaseName String
Human-readable name that identifies the database.
policyItemId String
projectId String
The unique ID for the project to create a data lake pipeline.
type String
Type of ingestion source of this Data Lake Pipeline.

DataLakePipelineTransformation
, DataLakePipelineTransformationArgs

Field string
Type string
Type of ingestion source of this Data Lake Pipeline.
Field string
Type string
Type of ingestion source of this Data Lake Pipeline.
field String
type String
Type of ingestion source of this Data Lake Pipeline.
field string
type string
Type of ingestion source of this Data Lake Pipeline.
field str
type str
Type of ingestion source of this Data Lake Pipeline.
field String
type String
Type of ingestion source of this Data Lake Pipeline.

Import

Data Lake Pipeline can be imported using project ID, name of the data lake and name of the AWS s3 bucket, in the format project_id–name, e.g.

$ pulumi import mongodbatlas:index/dataLakePipeline:DataLakePipeline example 1112222b3bf99403840e8934--test-data-lake-pipeline-test
Copy

See MongoDB Atlas API Documentation for more information.

To learn more about importing existing cloud resources, see Importing resources.

Package Details

Repository
MongoDB Atlas pulumi/pulumi-mongodbatlas
License
Apache-2.0
Notes
This Pulumi package is based on the mongodbatlas Terraform Provider.
MongoDB Atlas v3.30.0 published on Friday, Mar 21, 2025 by Pulumi
pulumi/pulumi-mongodbatlas