I recently encountered a common problem: needing to download a large Google Drive folder (around 30 gigabytes!) containing numerous files and subfolders. Manually selecting and downloading each file would take forever. The “Download All” option wasn’t much better, as Google’s zipping process for such a large volume of data also consumes a significant amount of time. To complicate matters, the link owner could delete the files at any moment, creating a sense of urgency. My solution? A script to clone all files from the shared link directly to my Google Drive, allowing me to sync them to my computer at my convenience.
Initial Approaches & The Winning Solution
My first thought was to use gdown, a Python library designed for Google Drive downloads. However, I quickly discovered its limitation: it can only handle up to 50 files per link, making it unsuitable for my situation.
The successful solution came with Google Colab. If you’re unfamiliar with it, Google Colab is a hosted service by Google that allows you to run Python scripts online, leveraging Google’s infrastructure. This means you get direct access to Google resources like Google Cloud and, crucially for this task, Google Drive.
Step-by-Step Guide with Google Colab
Here’s how to use Google Colab to clone large Google Drive folders:
You’ll need to create three separate code blocks and run them sequentially.
Step 1: Authorize Google Colab
This step grants Google Colab permission to access your Google Drive, enabling it to copy files on your behalf.
1 2 3 4 5 6 7 8
# STEP 1: Authorize Google Colab to access your Google Drive. from google.colab import auth from googleapiclient.discovery import build from googleapiclient.errors import HttpError
# This will trigger a pop-up window for you to authorize access. auth.authenticate_user() drive_service = build('drive', 'v3')
Step 2: Define the Copy Functions
This code block defines the necessary functions to recursively copy folders and their contents from the source to your Google Drive.
# STEP 2: Define the functions to copy the folder. def get_folder_id_from_url(url): """Extracts the folder ID from a Google Drive folder URL.""" try: return url.split('/')[-1].split('?')[0] except: return None
def copy_folder_recursive(service, source_folder_id, destination_parent_id, new_folder_name): """Recursively copies a folder and its contents.""" print(f"Creating root folder: '{new_folder_name}'...") # Create the root folder in the destination file_metadata = { 'name': new_folder_name, 'mimeType': 'application/vnd.google-apps.folder', 'parents': [destination_parent_id] } try: new_root_folder = service.files().create(body=file_metadata, fields='id').execute() new_root_folder_id = new_root_folder.get('id') print(f"Successfully created root folder with ID: {new_root_folder_id}") _copy_children(service, source_folder_id, new_root_folder_id) return new_root_folder_id except HttpError as error: print(f"An error occurred while creating the root folder: {error}") return None
def _copy_children(service, source_folder_id, destination_folder_id): """Helper function to copy contents of a folder.""" page_token = None while True: try: response = service.files().list(q=f"'{source_folder_id}' in parents and trashed=false", spaces='drive', fields='nextPageToken, files(id, name, mimeType)', pageToken=page_token).execute() for file in response.get('files', []): print(f"Found item: {file.get('name')} ({file.get('mimeType')})") if file.get('mimeType') == 'application/vnd.google-apps.folder': # It's a folder, create it in the destination and recurse print(f"Creating sub-folder: {file.get('name')}") folder_metadata = { 'name': file.get('name'), 'mimeType': 'application/vnd.google-apps.folder', 'parents': [destination_folder_id] } new_folder = service.files().create(body=folder_metadata, fields='id').execute() print(f"Copying contents of '{file.get('name')}'...") _copy_children(service, file.get('id'), new_folder.get('id')) else: # It's a file, copy it print(f"Copying file: {file.get('name')}") file_metadata = { 'name': file.get('name'), 'parents': [destination_folder_id] } service.files().copy(fileId=file.get('id'), body=file_metadata).execute() page_token = response.get('nextPageToken', None) if page_token is None: break except HttpError as error: print(f'An error occurred: {error}') break
Step 3: Run the Main Process
Finally, execute this code block to initiate the cloning process. You’ll be prompted to enter the Google Drive folder URL and a name for your new folder.
# STEP 3: Run the main process. def main(): """Main function to run the folder copy process.""" source_folder_url = input("Enter the Google Drive folder URL to clone: ") source_folder_id = get_folder_id_from_url(source_folder_url)
if not source_folder_id: print("Invalid Google Drive folder URL.") return
new_folder_name = input("Enter the name for your new folder in 'My Drive': ")
# 'root' is a special alias for the main "My Drive" folder destination_parent_id = 'root'
if new_folder_id: print(f"✅ Successfully cloned folder! You can find '{new_folder_name}' in your Google Drive.") else: print("❌ Folder cloning failed.")
# Run the script main()
Conclusion
By following these steps, you can efficiently clone large Google Drive folders directly to your own Google Drive using Google Colab. The “magic” happens in Step 1, where you grant Colab the necessary permissions. This method bypasses the limitations of traditional download methods and ensures you can secure your files quickly.
I hope this guide helps you manage your large Google Drive downloads with ease! Let me know in the comments if you have any questions or run into issues.
I uploaded the VMWARE Disk to my Proxmox, in this case, I put it in /root/
After that, I log to my Proxmox using SSH. Remember that, if you are trying to create a VM in Promox with ID 111, you’ll use this number in all this tutorial. In my case, it’s 111. Secondly, I’m using zfs, so my storage is local-zfs. You may check your storage by using cmd pvesm status
1 2 3 4
cd /var/lib/vz/images/ mkdir 111 cd 111 qm importdisk 111 /root/1ClickJXv2-cl3.vmdk local-zfs
I’ll got this result like:
1 2 3 4 5 6 7 8 9 10 11 12 13 14
root@lenovo:/var/lib/vz/images/111# qm importdisk 111 /root/1ClickJXv2-cl3.vmdk local-zfs importing disk '/root/1ClickJXv2-cl3.vmdk' to VM 111 ... transferred 0.0 B of 500.0 GiB (0.00%) transferred 5.0 GiB of 500.0 GiB (1.00%) transferred 10.0 GiB of 500.0 GiB (2.00%) transferred 15.0 GiB of 500.0 GiB (3.00%) transferred 20.0 GiB of 500.0 GiB (4.00%) transferred 25.0 GiB of 500.0 GiB (5.00%) transferred 30.0 GiB of 500.0 GiB (6.00%) transferred 35.0 GiB of 500.0 GiB (7.00%) transferred 40.0 GiB of 500.0 GiB (8.00%) transferred 45.0 GiB of 500.0 GiB (9.00%) transferred 50.0 GiB of 500.0 GiB (10.00%) ...
Part 2: Create VM
Now I’m going to create a VM in Promox, set the VMID is 111. Do not add Disk while creating.
Back to ssh and run:
1 2
qm set 111 --scsihw virtio-scsi-pci --scsi0 local-zfs:vm-111-disk-0 qm set 111 --boot order=scsi0
Part 3: Fake VM Machine
Some part of server doesnot support running under virtual machine. So you need to make the machine looks like a real machine.
1 2 3 4 5
qm set 111 --args "-cpu host,-hypervisor -device isa-debug-exit,iobase=0xf4,iosize=0x04" qm set 111 --vga virtio qm set 111 --net0 virtio=00:16:3E:12:34:56,bridge=vmbr0 qm set 111 --smbios1 manufacturer=HP,product=EliteBook qm set 111 --machine pc
Check QM Config by cmd qm config 111. Results should be:
Testing Your Luck: Generating an Ethereum Wallet and Checking Its Balance
Have you ever wondered what it would be like to randomly generate an Ethereum wallet and check if it has any balance? Well, let’s be honest—it’s a near-impossible dream, but that doesn’t mean we can’t have some fun testing our luck!
In this blog post, we’ll explore a simple Python script that generates an Ethereum wallet and checks its balance. If you’re the luckiest person in the world, who knows? You might just stumble upon an old wallet with some forgotten ETH!
Setting Up Your Environment
Before running the script, ensure you have Python installed along with the required dependencies. You can install the necessary package using:
1
pip install eth_account --break-system-packages
This will install the eth_account package, which allows us to generate Ethereum wallets programmatically.
Bypassing Public Node Rate Limits
If you want to check balances at scale, relying on public Ethereum nodes isn’t practical since they often limit requests to 5-10 per second. You might consider running your own Ethereum node using services like Infura, Alchemy, or a self-hosted Geth node. This way, you can perform unlimited checks without hitting request limits. Check my lastest post for the tutorial to install Ethereum node. In this example, the IP of Ethereum node is: 192.168.1.150
The Ethereum Wallet Generator
Here’s a simple Python script that generates an Ethereum private key, derives the corresponding public address, and checks the balance:
import requests import json import secrets import threading import time from eth_account import Account
defgenerate_eth_account(): """ Generate a new Ethereum account with a private key. :return: Tuple (private_key, address) """ private_key = "0x" + secrets.token_hex(32) account = Account.from_key(private_key) return private_key, account.address
defget_eth_balance(node_url, address): """ Fetch the Ethereum balance of a given address from a JSON-RPC node. :param node_url: The URL of the Ethereum node (e.g., http://192.168.1.150:8080) :param address: The Ethereum address to query :return: Balance in Ether (float) """ payload = { "jsonrpc": "2.0", "method": "eth_getBalance", "params": [address, "latest"], "id": 1 } headers = {"Content-Type": "application/json"} try: response = requests.post(node_url, headers=headers, data=json.dumps(payload)) response_data = response.json() if"result"in response_data: wei_balance = int(response_data["result"], 16) ether_balance = wei_balance / 1e18 return ether_balance else: return0.0 except requests.exceptions.RequestException as e: return0.0
global_counter = 0 good = 0 counter_lock = threading.Lock() start_time = time.time() num_threads = 300# Adjust the number of threads as needed
defprocess_account(node_url): """ Generate an Ethereum account, get its balance, and save results to files. Runs indefinitely. """ global global_counter, num_threads, good, start_time whileTrue: private_key, address = generate_eth_account() balance = get_eth_balance(node_url, address) log_entry = f"{address}{private_key}{balance}\n" withopen("log.txt", "a") as log_file: log_file.write(log_entry) if balance != 0.0: withopen("good.txt", "a") as good_file: good_file.write(log_entry) good += 1 with counter_lock: global_counter += 1 if global_counter % 1000 == 0: elapsed_time = time.time() - start_time print(f"{global_counter} checked | Runtime: {elapsed_time:.2f} seconds | Total {num_threads} threads | Total {good} good") start_time = time.time()
defmain(): node_url = "http://192.168.1.150:8080"
threads = [threading.Thread(target=process_account, args=(node_url,), daemon=True) for _ inrange(num_threads)] for thread in threads: thread.start() for thread in threads: thread.join()
if __name__ == "__main__": main()
Why Is This Fun but Unrealistic?
Ethereum addresses are derived from a 256-bit private key space, meaning there are 2^256 possible addresses. The chances of generating an address that already has ETH are astronomically small—essentially impossible. But hey, it’s still fun to dream, right?
Final Thoughts
Give it a shot and let me know if you hit the jackpot (spoiler: you won’t). 😆 Happy coding!
Add the script tag to the after-footer template. Make sure to place it below the jQuery script tag. In my case, the file is located at: /var/www/myblog/themes/landscape/layout/_partial/after-footer.ejs
Hi there! I’m just a curious soul who loves dabbling in the world of IT. I enjoy learning about coding, design, and all things tech-related, but let’s be real — I’m more of a “copy and paste” kind of person than a hardcore developer. If there’s a snippet of code or a cool design trick out there, you can bet I’ll grab it, tweak it, and make it my own!
What really gets me excited is playing around with AI. I love using it to mess with computers — whether it’s generating something fun, automating a boring task, or just seeing what crazy things I can make happen. Technology is like a playground for me, and AI is my favorite toy.
So yeah, that’s me in a nutshell: a bit lazy, a lot curious, and always ready to experiment with the next cool thing I find online. Stick around if you’re into quick hacks and AI-powered adventures!
Code Disclaimer
Hey, listen up! Every single piece of code you see on this page? Yup, it’s all copy and paste. I didn’t write it from scratch — I snagged it from somewhere out there in the wild internet, tweaked it a bit, and bam, here it is.
Warning
Big fat warning: I’m not responsible if you take this code and use it for something shady or downright evil. You wanna build a cool thing? Awesome! You wanna cause chaos? Not on my watch — but also not my problem. Use it at your own risk!
So, feel free to grab whatever you like, play with it, break it, or make it better. Just don’t come crying to me if the world explodes because of a misplaced semicolon. Happy coding (or copying)!
Markdown is a lightweight markup language used to format plain text into structured documents, often used for writing documentation, README files, and web content.
Basic Markdown Syntax
Headings
Use # followed by a space for different heading levels: