Skip to content

Data behemoths: How large users and unique structures impact data migrations

The biggest obstacles to a swift data migration hide in plain sight. Large users and unique data structures can cause trouble for the migration process if you haven’t factored them into your strategy.

Here, we explore why large users and unique data structures influence migrations, and what you can do to mitigate the risks and keep business running smoothly.

Why do large users impact the data migration process?

Large users have high item count, big file size, or a combination of the two. How these item counts and file sizes interact with destination API restrictions and quota limits is what makes them hard to migrate.

During a migration, data is moved from the source to the destination platform. It’s encrypted in transit and at rest to ensure the data is secure throughout the process. The larger the file, the longer it takes to migrate. And API restrictions limit the size of the file you can download.

Quota limits applied by cloud providers can slow down your migration

Cloud providers also limit how many items can be migrated. This could mean two users with the same amount of data could migrate at different speeds – if one has a higher file count than the other. User A – with 10GB spread across 50 items – migrates faster than user B – who has 10GB and 100 items.  

Both Google Cloud Platform and Microsoft Azure have quota limits for how much data can be migrated at a time, and restrictions on how many API calls you can make.  A user with many small files can take longer to migrate than a user with fewer, larger files. 

It may be necessary to upgrade your tech

Your resource limitations, such as the number and specification of servers, also influence data migration project timelines. Those working to a hard deadline might choose to invest in more or better servers – but this isn’t always an option. 

What are unique data structures and how do they impact migration timelines?

When it comes to unique data structures, you’re looking for file types (MIME types) such as folders, movies, and high-resolution images. Large files that hold up migrations. 

A large user with a unique data structure could consist of a high number of images and movie files in high resolution. This combines big data size and item count, making for a very different migration compared to a company with lots of smaller data files like text documents and emails. 

Some companies have a low number of huge files, which also creates complexity. This causes bottlenecks when you have the resources to migrate items quickly, but your target environment has an upload limit. You could max out Google’s 750GB a day upload limit with a few files, in a few hours.

How to deal with large users

You can’t avoid large users – sooner or later you need to get them across to your target system. But what can you do to control their impact on the data migration process? 

  1. Quantify them. How many large users do you have, and how much data do they have? Migration tools can give you an accurate view of item count.
  2. Encourage large users to tidy up their files ahead of migration. Delete obsolete data to avoid hampering the migration process with old files.
  3. Plan your migration with large users in mind. Can you structure your data migration process so that recent files are migrated first? This way, you can get into the target destination quickly and migrate legacy data later.
  4. Opt not to migrate permissions on files. Migrate files minus the permissions if your priority is getting data into the target destination, fast.

Use CloudM’s environment and readiness scans to identify potential roadblocks

Modern migration tools can also help you deal with large users. Take CloudM environment scan, which gives you an item count, shared folder count, and other data points you can use to scope out the migration and estimate a timeline. 

Secondly, the CloudM readiness scan checks over source and destination compatibility and connection, flagging limitations and revealing environmental complexities. You can use this scan as a last-minute safety check when you’re ready to migrate. For hard deadlines – such as mergers and acquisitions – using a migration service ensures a frictionless, managed process.

CloudM can help you tame the biggest data behemoth

The right tools can help you avoid disruption and maintain business continuity throughout the data migration process. For those keen to tackle the migration themselves, CloudM’s consulting services give you tailored training and advice. 

For a fully managed project, our white-glove migration services ensure the process is handled with care and expertise. We’ve worked with large users, complex data structures, and huge data volumes. We can get you from source to target destination swiftly.

With CloudM Backup, you can backup the following Calendar data:

  • Events and meetings: We back up and restore meeting which include meeting links, including Zoom links. We do not back up events without meeting links.

Yes, you can back up recurring meetings with CloudM Backup.

We do not back up Tasks at the moment.

In Google Calendar, attachments are a link to a Drive item. We will back up the item if the user’s Drive is also being backed up and restore the meeting with the link included. The Drive file itself can be backed up separately if required.

These will be backed up as event attendees. Handling of edge cases, such as when a user tries to restore an event and the resource has since become occupied, will be handled by your Workspace administrator.

The default frequency for backing up Calendar is 8 hours.

Please check our knowledge base for detailed information on how to restore a

backup of Google Calendar.

About CloudM
CloudM is an award-winning SaaS company whose humble beginnings in Manchester have grown into a global business in just a few short years.

Our team of tech-driven innovators have designed a SaaS data management platform for you to get the most from your digital workspace. Whether it’s Microsoft 365, Google Workspace or other SaaS applications, CloudM drives your business through a simple, easy-to-use interface, helping you to work smarter, not harder.

By automating time-consuming tasks like IT admin, onboarding & offboarding, archiving and migrations, the CloudM platform takes care of the day-to-day, allowing you to focus on the big picture.

With over 35,000 customers including the likes of Spotify, Netflix and Uber, our all-in-one platform is putting office life on auto-pilot, saving you time, stress and money.

About Version 2 Digital

Version 2 Digital is one of the most dynamic IT companies in Asia. The company distributes a wide range of IT products across various areas including cyber security, cloud, data protection, end points, infrastructures, system monitoring, storage, networking, business productivity and communication products.

Through an extensive network of channels, point of sales, resellers, and partnership companies, Version 2 offers quality products and services which are highly acclaimed in the market. Its customers cover a wide spectrum which include Global 1000 enterprises, regional listed companies, different vertical industries, public utilities, Government, a vast number of successful SMEs, and consumers in various Asian cities.

Product release: Protect your Google Calendar from data loss with CloudM Backup 1.8

Google Calendar is an essential tool for any company using the Google Workspace productivity suite. Keeping its data safe from cyberattacks, human error and malicious users is imperative to ensure business continuity. Following its latest release, CloudM Backup helps you do just that. 

CloudM Backup 1.8: Improve business continuity by backing up your Google Calendar data

Google Calendar has quickly become an integral part of Google Workspace, including business, personal, public and holiday calendars. Enabling users to schedule events, add reminders, share calendars and access them from anywhere, the data it contains is vital for the running of your business. Not being able to access it temporarily is a nuisance. Losing the data forever can be catastrophic. 

Comprehensive protection for your most important GWS applications

CloudM Backup already protects your users’ Google Mail, Drive and Shared Drives data from data loss due to cyberattacks, malicious users, accidental deletion and more. The most recent release ensures that your Calendar data is now also safe from the consequences of data loss and can be recovered quickly to ensure business can continue. 

CloudM Backup ensures your Google Mail, Drive and Calendar data is protected from the consequences of data loss.

 

Also in this release: Increased stability when handling large backups
With Backup 1.8.1, which will come a week after 1.8., we’ve invested in improvements to how we handle large files. Now, users and drives with large amounts of data to be backed up will be handled much more effectively, minimising backup times and reducing errors to ensure a smooth backup experience for all customers.

Backing up Google Calendar: Frequently asked questions

With CloudM Backup, you can backup the following Calendar data:

  • Events and meetings: We back up and restore meeting which include meeting links, including Zoom links. We do not back up events without meeting links.

Yes, you can back up recurring meetings with CloudM Backup.

We do not back up Tasks at the moment.

In Google Calendar, attachments are a link to a Drive item. We will back up the item if the user’s Drive is also being backed up and restore the meeting with the link included. The Drive file itself can be backed up separately if required.

These will be backed up as event attendees. Handling of edge cases, such as when a user tries to restore an event and the resource has since become occupied, will be handled by your Workspace administrator.

The default frequency for backing up Calendar is 8 hours.

Please check our knowledge base for detailed information on how to restore a

backup of Google Calendar.

About CloudM
CloudM is an award-winning SaaS company whose humble beginnings in Manchester have grown into a global business in just a few short years.

Our team of tech-driven innovators have designed a SaaS data management platform for you to get the most from your digital workspace. Whether it’s Microsoft 365, Google Workspace or other SaaS applications, CloudM drives your business through a simple, easy-to-use interface, helping you to work smarter, not harder.

By automating time-consuming tasks like IT admin, onboarding & offboarding, archiving and migrations, the CloudM platform takes care of the day-to-day, allowing you to focus on the big picture.

With over 35,000 customers including the likes of Spotify, Netflix and Uber, our all-in-one platform is putting office life on auto-pilot, saving you time, stress and money.

About Version 2 Digital

Version 2 Digital is one of the most dynamic IT companies in Asia. The company distributes a wide range of IT products across various areas including cyber security, cloud, data protection, end points, infrastructures, system monitoring, storage, networking, business productivity and communication products.

Through an extensive network of channels, point of sales, resellers, and partnership companies, Version 2 offers quality products and services which are highly acclaimed in the market. Its customers cover a wide spectrum which include Global 1000 enterprises, regional listed companies, different vertical industries, public utilities, Government, a vast number of successful SMEs, and consumers in various Asian cities.

Secure your data in 5 steps: A quick guide to CloudM Backup

Remote work and cloud infrastructure have revolutionized productivity, but they’ve also introduced new risks. A larger digital footprint means more opportunities for cyberattacks, data loss, or accidental deletions. Prevention is critical, but when things go wrong, a robust backup system is your safety net. 

CloudM Backup offers a dedicated, automated, and compliant solution to safeguard your data in minutes. Here’s how to get started in five simple steps:

Step 1: Set up your storage bucket in Google Cloud Storage

Start by creating a secure destination for your data backups. A storage bucket in Google Cloud Storage is where your data will be saved.

To create your bucket, you’ll need:

  • A Google Cloud billing account
  • Permissions to create or manage a GCP project

TIP: Automate the setup process using our PowerShell script for faster, error-free configuration. Full instructions are available in our Knowledge Base.

If you already have a Google Cloud Storage bucket, you can skip to Step 2!

Step 2: Assign permissions to your Backup admin

Designate a backup admin to manage and restore your data. For more control, create a new admin role with specific permissions. Permissions can be assigned via the settings page of your CloudM account. 

Permissions to assign:

  • Edit Backup Configuration & Policies
  • Backup User Restoration & View Status
  • Backup Shared Drives Restoration & View Status

To manage Organizational Units (OUs) and Smart Teams, assign permissions such as: 

  • Manage OUs
  • Manage Smart Teams
  • Manage Smart Teams Priority
  • Manage Users
  • View OU Hierarchy

TIP: Limit admin access by creating a role scoped to specific Organizational Units (OUs). Assign the role to users via the ‘Assigned Users’ tab.

play_circle

 

Step 3: Enable domain-wide delegation for seamless access

Set up a domain-wide delegated service account to connect your Google Cloud Storage bucket and apply backup policies. This step ensures efficient API usage and uninterrupted access. Visit our knowledge base for detailed instructions.

Step 4: Connect your Google Cloud Storage bucket to CloudM Backup

With your bucket ready, link it to CloudM Backup for synchronization. Simply navigate to the storage configuration page, select your bucket, and provide a Service Account key.

TIP: Test your connection to confirm everything works smoothly before proceeding.

 

Step 5: Create and assign your Backup policy

Craft customized backup policies to define what gets backed up and for how long.

  • Navigate to ‘Create and Edit Policies’ and choose the policy type (Users or Shared Drives).
  • Assign policies to OUs, Smart Teams, or Shared Drives as needed. 

TIP: Regularly review and adjust your policies to stay compliant with evolving regulations.

Sit back and relax

With these five steps, you’re now backup-ready! CloudM Backup ensures your organization’s data is protected, compliant, and recoverable, providing peace of mind and reliable business continuity.

About CloudM
CloudM is an award-winning SaaS company whose humble beginnings in Manchester have grown into a global business in just a few short years.

Our team of tech-driven innovators have designed a SaaS data management platform for you to get the most from your digital workspace. Whether it’s Microsoft 365, Google Workspace or other SaaS applications, CloudM drives your business through a simple, easy-to-use interface, helping you to work smarter, not harder.

By automating time-consuming tasks like IT admin, onboarding & offboarding, archiving and migrations, the CloudM platform takes care of the day-to-day, allowing you to focus on the big picture.

With over 35,000 customers including the likes of Spotify, Netflix and Uber, our all-in-one platform is putting office life on auto-pilot, saving you time, stress and money.

About Version 2 Digital

Version 2 Digital is one of the most dynamic IT companies in Asia. The company distributes a wide range of IT products across various areas including cyber security, cloud, data protection, end points, infrastructures, system monitoring, storage, networking, business productivity and communication products.

Through an extensive network of channels, point of sales, resellers, and partnership companies, Version 2 offers quality products and services which are highly acclaimed in the market. Its customers cover a wide spectrum which include Global 1000 enterprises, regional listed companies, different vertical industries, public utilities, Government, a vast number of successful SMEs, and consumers in various Asian cities.

Delta-first or traditional? Choosing a migration approach

Choosing the right migration approach for your project

A successful migration starts long before you move the first file, and the consequences of insufficient planning and preparation can be disastrous. Back in 2019, British bank TSB locked millions of customers out of their accounts, following a failed IT migration. In 2022, the bank was fined over £48 million for its failure to manage the migration effectively. 

Three migration approaches: pros and cons

1. Traditional migration (waterfall methodology)

The traditional approach involves migrating bulk mail and drive items, followed by regular, smaller (delta) migrations of new and modified files. 

By moving bulk data upfront and capturing recent files with delta migrations, traditional migrations minimise downtime. There are some drawbacks – you need robust processes to track, detect, and sync delta migrations accurately. And the initial bulk migration is resource-intensive. 

Traditional migrations are linear, making it harder to change tactics once the project is in motion. This type of migration is a solid fit for high-volume systems with active users but might be too rigid for simple migrations or time-limited projects. 

2. Delta-first migration

Delta-first migrations are like traditional migrations in reverse. 

New and recently updated (delta) data is migrated first. Then historical (bulk) data is migrated later. It’s a less conventional approach but gives users immediate functionality with minimal operational impact.

Teams can move into the destination platform faster, and incremental delta migrations mean the bulk migration is smaller (and faster). 

There are limitations, though. You need to maintain source and destination platforms throughout the migration. Separating delta and bulk data requires robust tracking mechanisms to manage transfers. Dependencies between delta and bulk data mean certain files may not function properly until the full migration is complete, and you need to sync both data types continuously.

3. Google migration mould (domain switch)

A Google migration mould lets users keep their domain name when moving from an existing Google workspace to a new one. A company could benefit from this approach if they’re moving to a new workspace but need to keep their existing domain name and email addresses. 

There’s a specific methodology which requires temporary domains to do the bulk migration, and a domain switch during the delta migration. Google’s tailored advanced migration settings such as selective data transfers and preserved permissions for shared drives, calendars, and files make the process easier. 

There are drawbacks though – Google API limits restrict the number of daily data transfers, and certain types of data (such as comments in a document) might not be migrated. Using a cloud migration service can get you past some of these limitations.

How to know which migration approach is right

To determine the right approach for your next migration, we recommend you work backwards. The most pressing outcome of the migration will most likely dictate the approach:

Goal: Limited downtime and business continuity

If limited downtime and business continuity are your main focus, a traditional approach ensures historical data is available for your go-live date. You can migrate users simultaneously with a big bang cutover, and avoid downtime by scheduling the migration over a weekend or planned closure. 

Goal: Speed

If you’re looking for speed, choosing not to migrate permissions on folders and files, selecting a specific date range (e.g. the last 12 months’ data), and getting users to delete unwanted data can speed things up. 

Goal: Vacating incumbent platform immediately

If your priority is vacating the platform immediately, a delta migration that moves recent data – say, 30 days’ worth – means you can go live fast. Then, you can backfill your target destination with historical data post-cutover. 

Other factors that influence how quickly you can complete a migration include:  

  • Amount and type of data
  • Data per user 
  • Folder, file, and email count
  • Media (MIME) types 
  • A lack of documentation for legacy systems 
  • Ability to coexist in two different environments

Can you combine migration approaches?

Combining migration approaches is possible and may in some cases be necessary to work around more complex migration scenarios and conflicting goals. Here’s an example:

A multinational company wants to prioritize its specific users and teams in their migration. The company completes a bulk migration for non-critical users and shared drives during planned downtime. Then, the company completes a delta-first migration for critical teams and executives’ recent data (e.g. the last 30 days) into the target system for immediate access. Historical data is backfilled post-cutover to limit delays.

CloudM can help you make the best choice

You can count on CloudM Migrate for a smooth migration. Choose from white glove migration services where we take care of the migration for you. Or get tailored training and advice through our consulting services that equip your team with the skills to deliver frictionless migrations.

About CloudM
CloudM is an award-winning SaaS company whose humble beginnings in Manchester have grown into a global business in just a few short years.

Our team of tech-driven innovators have designed a SaaS data management platform for you to get the most from your digital workspace. Whether it’s Microsoft 365, Google Workspace or other SaaS applications, CloudM drives your business through a simple, easy-to-use interface, helping you to work smarter, not harder.

By automating time-consuming tasks like IT admin, onboarding & offboarding, archiving and migrations, the CloudM platform takes care of the day-to-day, allowing you to focus on the big picture.

With over 35,000 customers including the likes of Spotify, Netflix and Uber, our all-in-one platform is putting office life on auto-pilot, saving you time, stress and money.

About Version 2 Digital

Version 2 Digital is one of the most dynamic IT companies in Asia. The company distributes a wide range of IT products across various areas including cyber security, cloud, data protection, end points, infrastructures, system monitoring, storage, networking, business productivity and communication products.

Through an extensive network of channels, point of sales, resellers, and partnership companies, Version 2 offers quality products and services which are highly acclaimed in the market. Its customers cover a wide spectrum which include Global 1000 enterprises, regional listed companies, different vertical industries, public utilities, Government, a vast number of successful SMEs, and consumers in various Asian cities.

Choosing a software solution: Eight things to consider for the NIS2 directive

The Network and Information Securities Directive (NIS) 2 aims to increase the security and resilience of essential services and digital infrastructure in the European Union. It affects essential entities (including energy, transport, and banking) and important entities (such as postal, food, and manufacturing).  

The legislation came into force on 18 October 2024, following the creation of national laws based on the directive. Key requirements include risk assessment, management and prevention, incident reporting within 24 hours, and backup and disaster recovery.

Software can help mandated organisations maintain compliance and stay ahead of cybersecurity threats. Here’s what you should look for in NIS2 compliance software.

What do you need to keep in mind when choosing a new software solution?

Fines and punishments for NIS2 are stringent – up to 2% of global annual revenue. Staying compliant isn’t just a cybersecurity matter, it’s a financial one. 

CloudM helps you meet the NIS2 implementation requirements with Backup, our solution for preventing data loss and ensuring recovery and restoration. Backup keeps data protected in the background, while you’re free to focus on the business.

€10 million

Essential entities face fines of up to €10 million or 2% of global annual revenue.

€7 million

Important entities face fines of up to €7 million or 1.4% of global annual revenue

 

These software features help stay on track and ensure compliance with the NIS2 directive.

 

1. Automation

Under the NIS2 directive, organizations need to have regular backups so critical data can be restored following an incident. When this process is automated it reduces the risk of human error and enables business continuity. Features like automated event logging and detection also mean you’re in a better position to meet the 24-hour incident response time. 

 

2. Support

With just 24 hours to respond to an incident, additional support from your software vendor can be a lifeline.

Responsive, round-the-clock support helps you navigate incidents swiftly and effectively within 24 hours. Beyond incident support, modern software providers also offer guidance and training on how to use tools effectively, so you can ensure future NIS2 compliance and be prepared for what’s ahead. 

 

3. Security and compliance

Effective prevention can help you avoid the most harmful situations. Using end-to-end encryption, single sign-on, multi-factor authentication, and role-based access controls protects your data. Your software provider should also meet security standards, such as ISO 27001, which demonstrates they use data protection best practices.

 

4. Incident reporting 

You need fast and comprehensive incident reporting mechanisms to meet the 24-hour requirement for NIS2 compliance. 

Real-time monitoring, alerting, and automated logging of suspicious or unusual activity can help you achieve this. These features enable your security team to respond effectively before any damage is done – and means instead of searching for the specific cause, they can focus on solutions for recovery first.

 

5. Disaster recovery

As the spate of recent NHS cyberattacks prove, security breaches are a real threat, with serious consequences for organizations.

Data recovery tools help to ensure business continuity. Frequent, automated data backups enable you to recover the most recent file versions. Flexible restoration options – such as folder, user, and item-specific recovery – can be particularly helpful if you only need to complete a partial restoration. 

 

6. Scalability and flexibility

Organizational data grows as your business does, and the ability to handle more of it is essential for NIS2 compliance. Every piece of data your organization gains is something that could leak or be accessed by a cybercriminal. 

Finding software that’s scalable and flexible will help you maintain business continuity. Whether you’re responding to regulatory or organizational changes, having software that grows with you and updates in line with new legislation is key. 

 

7. Data protection and privacy

Strict data retention controls – including the ability to wipe data as needed and limit access to business-critical data – can help you meet NIS2 directive risk management requirements.

Security requirements differ in every region, and your software should be able to accommodate these variations. Consider the range of security standards your organization needs to meet – and whether your chosen provider is set up for future changes. 

 

8. Customizable security settings

All organizations require different levels of data access. Getting this right can help you meet NIS2 risk management, incident prevention, and corporate accountability requirements.

Opt for software with adjustable access levels and protocols depending on organization size and the amount of data. By assigning specific permissions to people in your organization, you can limit the risk of data falling into the wrong hands.

 

About CloudM
CloudM is an award-winning SaaS company whose humble beginnings in Manchester have grown into a global business in just a few short years.

Our team of tech-driven innovators have designed a SaaS data management platform for you to get the most from your digital workspace. Whether it’s Microsoft 365, Google Workspace or other SaaS applications, CloudM drives your business through a simple, easy-to-use interface, helping you to work smarter, not harder.

By automating time-consuming tasks like IT admin, onboarding & offboarding, archiving and migrations, the CloudM platform takes care of the day-to-day, allowing you to focus on the big picture.

With over 35,000 customers including the likes of Spotify, Netflix and Uber, our all-in-one platform is putting office life on auto-pilot, saving you time, stress and money.

About Version 2 Digital

Version 2 Digital is one of the most dynamic IT companies in Asia. The company distributes a wide range of IT products across various areas including cyber security, cloud, data protection, end points, infrastructures, system monitoring, storage, networking, business productivity and communication products.

Through an extensive network of channels, point of sales, resellers, and partnership companies, Version 2 offers quality products and services which are highly acclaimed in the market. Its customers cover a wide spectrum which include Global 1000 enterprises, regional listed companies, different vertical industries, public utilities, Government, a vast number of successful SMEs, and consumers in various Asian cities.

×

Hello!

Click one of our contacts below to chat on WhatsApp

Social Chat is free, download and try it now here!

×