代码之家 › 专栏 › 技术社区 › Dan

在Google App Engine Flex上运行时,使用Java将文件从Google存储桶移动到Google Drive

google-cloud-storage google-drive-api google-cloud-platform google-app-engine java

Dan · 技术社区 · 6 年前

我正在Google App Engine Flex环境中运行一个应用程序。我的目标是允许文件从应用程序上传到Google驱动器。由于应用程序引擎没有文件系统(或我真正理解的文件系统),谷歌的文档中说要上传到存储桶:

https://cloud.google.com/appengine/docs/flexible/java/using-cloud-storage

我的代码工作得很好-上传到存储桶。谷歌驱动器代码也工作得很好。

我的问题是-如何将存储桶文件上传到Google驱动器?

我发现了一些帖子,这些帖子建议我必须从存储桶下载文件,据我所知,这些文件必须保存到一个文件系统,然后由谷歌驱动器代码提取并上传到谷歌驱动器。

copy file from Google Drive to Google Cloud Storage within Google

How to download a file from Google Cloud Storage with Java?

我要做的就是 java.io.File 对象从存储桶下载。我有 blob.getMediaLink() 从Google存储代码返回-是否可以使用该代码而不是下载?

谷歌存储代码:

public static String sendFileToBucket(InputStream fileStream, String fileName) throws IOException {
    Logger.info("GoogleStorage: sendFileToBucket: Starting...");

    GoogleCredential credential = null;
    String credentialsFileName = "";
    Storage storage = null;
    Blob blob = null;
    List<Acl> acls = null;

    // credential = authorize();

    // storage = StorageOptions.getDefaultInstance().getService();

    try {
        Logger.info("GoogleStorage: sendFileToBucket: Getting credentialsFileName path...");
        credentialsFileName = Configuration.root().getString("google.storage.credentials.file");
        Logger.info("GoogleStorage: sendFileToBucket: credentialsFileName = " + credentialsFileName);

        Logger.info("GoogleStorage: sendFileToBucket: Setting InputStream...");
        InputStream in = GoogleStorage.class.getClassLoader().getResourceAsStream(credentialsFileName);
        if (in == null) {
            Logger.info("GoogleStorage: sendFileToBucket: InputStream is null");
        }
        Logger.info("GoogleStorage: sendFileToBucket: InputStream set...");

        try {
            storage = StorageOptions.newBuilder().setCredentials(ServiceAccountCredentials.fromStream(in)).build()
                    .getService();
        } catch (StorageException se) {
            System.out.println("--- START ERROR WITH SETTING STORAGE OBJECT ---");
            se.printStackTrace();
            System.out.println("--- END ERROR WITH SETTING STORAGE OBJECT ---");
        }

        // Modify access list to allow all users with link to read file
        acls = new ArrayList<>();
        acls.add(Acl.of(Acl.User.ofAllUsers(), Acl.Role.READER));

        try {
            Logger.info("GoogleStorage: sendFileToBucket: Setting Blob object...");
            blob = storage.create(BlobInfo.newBuilder(BUCKET_NAME, fileName).setAcl(acls).build(), fileStream);
            Logger.info("GoogleStorage: sendFileToBucket: Blob Object set...");
        } catch (StorageException se) {
            System.out.println("--- START ERROR WITH SETTING BLOB OBJECT ---");
            se.printStackTrace();
            System.out.println("--- END ERROR WITH SETTING BLOB OBJECT ---");
        }
    } catch (IOException ex) {
        System.out.println("--- START ERROR SENDFILETOBUCKET ---");
        ex.printStackTrace();
        System.out.println("--- END ERROR SENDFILETOBUCKET ---");
    }
    Logger.info("GoogleStorage: sendFileToBucket: blob.getMediaLink() = " + blob.getMediaLink());
    return blob.getMediaLink();
}

Google驱动器代码:

public static String uploadFile(java.io.File file, String folderIDToFind) throws IOException {
    String fileID = "";
    String fileName = "";
    try {
        Logger.info("GoogleDrive: uploadFile: Starting File Upload...");
        // Build a new authorized API client service.
        Drive service = getDriveService();
        Logger.info("GoogleDrive: uploadFile: Completed Drive Service...");

        // Set the folder...
        String folderID = Configuration.root().getString("google.drive.folderid");
        Logger.info("GoogleDrive: uploadFile: Folder ID = " + folderID);

        String folderIDToUse = getSubfolderID(service, folderID, folderIDToFind);

        String fullFilePath = file.getAbsolutePath();
        Logger.info("GoogleDrive: uploadFile: Full File Path: " + fullFilePath);
        File fileMetadata = new File();

        // Let's see what slashes exist to get the correct file name...
        if (fullFilePath.contains("/")) {
            fileName = StringControl.rightBack(fullFilePath, "/");
        } else {
            fileName = StringControl.rightBack(fullFilePath, "\\");
        }
        String fileContentType = getContentType(fileName);
        Logger.info("GoogleDrive: uploadFile: File Content Type: " + fileContentType);
        fileMetadata.setName(fileName);
        Logger.info("GoogleDrive: uploadFile: File Name = " + fileName);

        Logger.info("GoogleDrive: uploadFile: Setting the folder...");
        fileMetadata.setParents(Collections.singletonList(folderIDToUse));
        Logger.info("GoogleDrive: uploadFile: Folder set...");

        // Team Drive settings...
        fileMetadata.set("supportsTeamDrives", true);

        FileContent mediaContent = new FileContent(fileContentType, file);

        File fileToUpload = service.files().create(fileMetadata, mediaContent).setSupportsTeamDrives(true)
                .setFields("id, parents").execute();

        fileID = fileToUpload.getId();
        Logger.info("GoogleDrive: uploadFile: File ID: " + fileID);
    } catch (Exception ex) {
        System.out.println(ex.toString());
        ex.printStackTrace();
    }
    Logger.info("GoogleDrive: uploadFile: Ending File Upload...");
    return fileID;
}

在上面的帖子中,我看到了如何获取存储桶文件,但如何将其传递给 java.io.file 对象:

How to download a file from Google Cloud Storage with Java?

感谢您的帮助。

2 回复 | 直到 6 年前

Ying Li 6 年前

您需要从GCS(谷歌云存储)将文件下载到本地计算机上,然后让Java代码将其提取并发送到Google Drive。

可以使用代码的第二部分。但您的第一部分实际上是将一个文件上载到GCS,然后获取一个服务URL。我想你不想那样。

而是调查 downloading from GCS ,然后运行代码的第二部分并将该文件上载到Google Drive。

===

如果您正在使用App Engine,并且不想使用本地计算机执行上载,则可以尝试以下操作:

Read from GCS and get the file to outputStream in the HTTP response 。那么你可以 upload the file to Google Drive 。有一些代码需要添加,您需要确保得到的字节流已转换为java。io。文件

Dan 6 年前

多亏了@Ying Li,我终于找到了解决办法。我最终创造了一个临时 java.io.File 使用 blob.getMediaLink() 上载到存储桶后返回的URL。以下是我创建的方法:

public static java.io.File createFileFromURL(String fileURL, String fileName) throws IOException {

    java.io.File tempFile = null;

    try {
        URL url = new URL(fileURL);
        Logger.info("GoogleControl: createFileFromURL: url = " + url);
        Logger.info("GoogleControl: createFileFromURL: fileName = " + fileName);
        String filePrefix = StringControl.left(fileName, ".") + "_";
        String fileExt = "." + StringControl.rightBack(fileName, ".");
        Logger.info("GoogleControl: createFileFromURL: filePrefix = " + filePrefix);
        Logger.info("GoogleControl: createFileFromURL: fileExt = " + fileExt);

        tempFile = java.io.File.createTempFile(filePrefix, fileExt);

        tempFile.deleteOnExit();
        FileUtils.copyURLToFile(url, tempFile);
        Logger.info("GoogleControl: createFileFromURL: tempFile name = " + tempFile.getName());
    } catch (Exception ex) {
        System.out.println(ex.toString());
        ex.printStackTrace();
    }

    return tempFile;
}

因此,整个类如下所示:

package google;

import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.net.URL;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collections;
import java.util.List;

import org.apache.commons.io.FileUtils;

import com.google.api.client.googleapis.auth.oauth2.GoogleCredential;
import com.google.api.client.googleapis.javanet.GoogleNetHttpTransport;
import com.google.api.client.http.FileContent;
import com.google.api.client.http.HttpTransport;
import com.google.api.client.json.JsonFactory;
import com.google.api.client.json.jackson2.JacksonFactory;
import com.google.api.client.util.store.FileDataStoreFactory;
import com.google.api.services.drive.Drive;
import com.google.api.services.drive.DriveScopes;
import com.google.api.services.drive.model.File;
import com.google.api.services.drive.model.FileList;
import com.google.auth.oauth2.*;
import com.google.cloud.storage.*;

import controllers.GlobalUtilities.StringControl;
import play.Configuration;
import play.Logger;

/**
 * @author Dan Zeller
 *
 */

public class GoogleControl {

    private static final String APPLICATION_NAME = "PTP";
    private static final JsonFactory JSON_FACTORY = JacksonFactory.getDefaultInstance();
    private static HttpTransport HTTP_TRANSPORT;
    private static final String BUCKET_NAME = Configuration.root().getString("google.storage.bucket.name");
    public static final String FILE_PREFIX = "stream2file";
    public static final String FILE_SUFFIX = ".tmp";

    static {
        try {
            HTTP_TRANSPORT = GoogleNetHttpTransport.newTrustedTransport();
        } catch (Throwable t) {
            t.printStackTrace();
            System.exit(1);
        }
    }

    @SuppressWarnings("deprecation")
    public static GoogleCredential authorize() throws IOException {
        GoogleCredential credential = null;
        String credentialsFileName = "";
        try {
            Logger.info("GoogleControl: authorize: Starting...");

            Logger.info("GoogleControl: authorize: Getting credentialsFileName path...");
            credentialsFileName = Configuration.root().getString("google.drive.credentials.file");
            Logger.info("GoogleControl: authorize: credentialsFileName = " + credentialsFileName);

            Logger.info("GoogleControl: authorize: Setting InputStream...");
            InputStream in = GoogleControl.class.getClassLoader().getResourceAsStream(credentialsFileName);
            if (in == null) {
                Logger.info("GoogleControl: authorize: InputStream is null");
            }
            Logger.info("GoogleControl: authorize: InputStream set...");

            Logger.info("GoogleControl: authorize: Setting credential...");
            credential = GoogleCredential.fromStream(in, HTTP_TRANSPORT, JSON_FACTORY)
                    .createScoped(Collections.singleton(DriveScopes.DRIVE));
        } catch (IOException ex) {
            System.out.println(ex.toString());
            System.out.println("Could not find file " + credentialsFileName);
            ex.printStackTrace();
        }
        Logger.info("GoogleControl: authorize: Ending...");
        return credential;
    }

    public static java.io.File createFileFromURL(String fileURL, String fileName) throws IOException {

        java.io.File tempFile = null;

        try {
            URL url = new URL(fileURL);
            Logger.info("GoogleControl: createFileFromURL: url = " + url);
            Logger.info("GoogleControl: createFileFromURL: fileName = " + fileName);
            String filePrefix = StringControl.left(fileName, ".") + "_";
            String fileExt = "." + StringControl.rightBack(fileName, ".");
            Logger.info("GoogleControl: createFileFromURL: filePrefix = " + filePrefix);
            Logger.info("GoogleControl: createFileFromURL: fileExt = " + fileExt);

            tempFile = java.io.File.createTempFile(filePrefix, fileExt);

            tempFile.deleteOnExit();
            FileUtils.copyURLToFile(url, tempFile);
            Logger.info("GoogleControl: createFileFromURL: tempFile name = " + tempFile.getName());
        } catch (Exception ex) {
            System.out.println(ex.toString());
            ex.printStackTrace();
        }

        return tempFile;
    }

    public static void downloadFile(String fileID) {
        // Set the drive service...
        Drive service = null;
        try {
            service = getDriveService();
        } catch (IOException e) {
            e.printStackTrace();
        }
        OutputStream outputStream = new ByteArrayOutputStream();
        try {
            service.files().get(fileID).executeMediaAndDownloadTo(outputStream);
        } catch (IOException e) {
            e.printStackTrace();
        }
    }

    public static String getContentType(String filePath) throws Exception {
        String type = "";
        try {
            Path path = Paths.get(filePath);
            type = Files.probeContentType(path);
            System.out.println(type);
        } catch (Exception ex) {
            System.out.println(ex.toString());
            ex.printStackTrace();
        }
        return type;
    }

    public static Drive getDriveService() throws IOException {
        Logger.info("GoogleControl: getDriveService: Starting...");
        GoogleCredential credential = null;
        Drive GoogleControl = null;
        try {
            credential = authorize();
            Logger.info("GoogleControl: getDriveService: Credentials set...");
            try {
                GoogleControl = new Drive.Builder(HTTP_TRANSPORT, JSON_FACTORY, credential)
                        .setApplicationName(APPLICATION_NAME).build();
            } catch (Exception ex) {
                System.out.println(ex.toString());
                ex.printStackTrace();
            }

        } catch (IOException ex) {
            System.out.println(ex.toString());
            ex.printStackTrace();
        }
        return GoogleControl;
    }

    public static String getPath() {
        String s = GoogleControl.class.getName();
        int i = s.lastIndexOf(".");
        if (i > -1)
            s = s.substring(i + 1);
        s = s + ".class";
        System.out.println("Class Name: " + s);
        Object testPath = GoogleControl.class.getResource(s);
        System.out.println("Current Path: " + testPath);
        return "";
    }

    public static String getSubfolderID(Drive service, String parentFolderID, String folderKeyToGet) {
        // We need to see if the folder exists based on the ID...
        String folderID = "";
        Boolean foundFolder = false;
        FileList result = null;
        File newFolder = null;

        // Set the drive query...
        String driveQuery = "mimeType='application/vnd.google-apps.folder' and '" + parentFolderID
                + "' in parents and name='" + folderKeyToGet + "' and trashed=false";

        try {
            result = service.files().list().setQ(driveQuery).setIncludeTeamDriveItems(true).setSupportsTeamDrives(true)
                    .execute();
        } catch (IOException e) {
            e.printStackTrace();
        }
        for (File folder : result.getFiles()) {
            System.out.printf("Found folder: %s (%s)\n", folder.getName(), folder.getId());
            foundFolder = true;
            folderID = folder.getId();
        }
        if (foundFolder != true) {
            // Need to create the folder...
            File fileMetadata = new File();
            fileMetadata.setName(folderKeyToGet);
            fileMetadata.setTeamDriveId(parentFolderID);
            fileMetadata.set("supportsTeamDrives", true);
            fileMetadata.setMimeType("application/vnd.google-apps.folder");
            fileMetadata.setParents(Collections.singletonList(parentFolderID));

            try {
                newFolder = service.files().create(fileMetadata).setSupportsTeamDrives(true).setFields("id, parents")
                        .execute();
            } catch (IOException e) {
                e.printStackTrace();
            }
            // Send back the folder ID...
            folderID = newFolder.getId();
            System.out.println("Folder ID: " + newFolder.getId());
        }

        return folderID;
    }

    @SuppressWarnings("deprecation")
    public static java.io.File sendFileToBucket(InputStream fileStream, String fileName) throws Exception {
        Logger.info("GoogleControl: sendFileToBucket: Starting...");

        GoogleCredential credential = null;
        String credentialsFileName = "";
        String outputFileName = "";
        Storage storage = null;
        Blob blob = null;
        List<Acl> acls = null;
        java.io.File returnFile = null;

        try {
            Logger.info("GoogleControl: sendFileToBucket: Getting credentialsFileName path...");
            credentialsFileName = Configuration.root().getString("google.storage.credentials.file");
            Logger.info("GoogleControl: sendFileToBucket: credentialsFileName = " + credentialsFileName);

            Logger.info("GoogleControl: sendFileToBucket: Setting InputStream...");
            InputStream in = GoogleControl.class.getClassLoader().getResourceAsStream(credentialsFileName);
            if (in == null) {
                Logger.info("GoogleControl: sendFileToBucket: InputStream is null");
            }
            Logger.info("GoogleControl: sendFileToBucket: InputStream set...");

            try {
                storage = StorageOptions.newBuilder().setCredentials(ServiceAccountCredentials.fromStream(in)).build()
                        .getService();
            } catch (Exception se) {
                System.out.println("--- START ERROR WITH SETTING STORAGE OBJECT ---");
                se.printStackTrace();
                System.out.println("--- END ERROR WITH SETTING STORAGE OBJECT ---");
            }

            // Modify access list to allow all users with link to read file
            acls = new ArrayList<>();
            acls.add(Acl.of(Acl.User.ofAllUsers(), Acl.Role.READER));

            try {
                Logger.info("GoogleControl: sendFileToBucket: Setting Blob object...");
                blob = storage.create(BlobInfo.newBuilder(BUCKET_NAME, fileName).setAcl(acls).build(), fileStream);
                Logger.info("GoogleControl: sendFileToBucket: Blob Object set...");
            } catch (Exception se) {
                System.out.println("--- START ERROR WITH SETTING BLOB OBJECT ---");
                se.printStackTrace();
                System.out.println("--- END ERROR WITH SETTING BLOB OBJECT ---");
            }

            Logger.info("GoogleControl: sendFileToBucket: blob.getMediaLink() = " + blob.getMediaLink());

            // Let's build a java.io.file to send back...
            returnFile = createFileFromURL(blob.getMediaLink(), fileName);

        } catch (Exception ex) {
            System.out.println("--- START ERROR SENDFILETOBUCKET ---");
            ex.printStackTrace();
            System.out.println("--- END ERROR SENDFILETOBUCKET ---");
        }
        return returnFile;
        // return outputFileName;
    }

    public static String uploadFile(java.io.File file, String folderIDToFind) throws IOException {
        String fileID = "";
        String fileName = "";
        try {
            Logger.info("GoogleControl: uploadFile: Starting File Upload...");
            // Build a new authorized API client service.
            Drive service = getDriveService();
            Logger.info("GoogleControl: uploadFile: Completed Drive Service...");

            // Set the folder...
            String folderID = Configuration.root().getString("google.drive.folderid");
            Logger.info("GoogleControl: uploadFile: Folder ID = " + folderID);

            String folderIDToUse = getSubfolderID(service, folderID, folderIDToFind);

            String fullFilePath = file.getAbsolutePath();
            Logger.info("GoogleControl: uploadFile: Full File Path: " + fullFilePath);
            File fileMetadata = new File();

            // Let's see what slashes exist to get the correct file name...
            if (fullFilePath.contains("/")) {
                fileName = StringControl.rightBack(fullFilePath, "/");
            } else {
                fileName = StringControl.rightBack(fullFilePath, "\\");
            }
            String fileContentType = getContentType(fileName);
            Logger.info("GoogleControl: uploadFile: File Content Type: " + fileContentType);
            fileMetadata.setName(fileName);
            Logger.info("GoogleControl: uploadFile: File Name = " + fileName);

            Logger.info("GoogleControl: uploadFile: Setting the folder...");
            fileMetadata.setParents(Collections.singletonList(folderIDToUse));
            Logger.info("GoogleControl: uploadFile: Folder set...");

            // Team Drive settings...
            fileMetadata.set("supportsTeamDrives", true);

            FileContent mediaContent = new FileContent(fileContentType, file);

            File fileToUpload = service.files().create(fileMetadata, mediaContent).setSupportsTeamDrives(true)
                    .setFields("id, parents").execute();

            fileID = fileToUpload.getId();
            Logger.info("GoogleControl: uploadFile: File ID: " + fileID);
        } catch (Exception ex) {
            System.out.println(ex.toString());
            ex.printStackTrace();
        }
        Logger.info("GoogleControl: uploadFile: Ending File Upload...");
        return fileID;
    }

}

下面是启动这一切的代码片段:

try {
    Logger.info("AdultPTPController: Starting File Upload...");
    File file = null;
    File fileFinal = null;
    String fileName = "";
    String fileContentType = "";
    String filePath = "";
    String fileID = "";
    String bucketFilePath = "";
    String bucketFileName = "";
    // String folderIDToFind =
    // adultDemo.getNew_Provider_Id().toString();
    String folderIDToFind = adultDemo.getLegacy_Provider_Id().toString();
    Http.MultipartFormData<File> formData = request().body().asMultipartFormData();
    if (formData != null) {
        Http.MultipartFormData.FilePart<File> filePart = formData.getFile("fileAttach");
        if (filePart != null) {
            fileName = filePart.getFilename().trim();
            // Is there a file?
            if (!fileName.equals("") && fileName != null) {
                Logger.info("AdultPTPController: File Name = " + fileName);
                fileContentType = filePart.getContentType();
                file = filePart.getFile();
                long size = Files.size(file.toPath());
                String fullFilePath = file.getPath();
                InputStream fileStream = new FileInputStream(fullFilePath);
                // Send the file/multipart content to the storage
                // bucket...
                Logger.info("AdultPTPController: Sending file to GoogleStorage - sendFileToBucket");
                File bucketFile = GoogleControl.sendFileToBucket(fileStream, fileName);
                //File bucketFile = null;
                Logger.info("AdultPTPController: File Sent to GoogleStorage - sendFileToBucket");

                // Send the file to Google Drive...
                if (bucketFile != null) {
                    // Upload the file and return the file ID...                            
                    Logger.info("AdultPTPController: File is not null, sending to uploadFile...");
                    bucketFileName = bucketFile.getName();
                    Logger.info("AdultPTPController: bucketFileName = " + bucketFileName);
                    fileID = GoogleControl.uploadFile(bucketFile, folderIDToFind);
                    Logger.info("AdultPTPController: File ID = " + fileID);
                    if (!fileID.equals("")) {
                        // Success...
                        // Create a file record...
                        FileUpload fileUpload = new FileUpload();
                        // Create a unique ID...
                        fileUpload.setFileKey(fileUpload.createFileKey());
                        // Set the needed fields...
                        fileUpload.setRecordID(adultDemo.getLegacy_Provider_Id().toString());
                        fileUpload.setRecordKey(adultPTP.getPtpkey());
                        fileUpload.setCreatedBy(user.getFullname());
                        fileUpload.setCreatedByEmail(user.getEmail());
                        fileUpload.setCreatedByKey(user.getUserkey());
                        fileUpload.setCreatedDate(GlobalUtilities.getCurrentLocalDateTime());
                        fileUpload.setDateCreatedDisplay(
                                GlobalUtilities.getStringDate(fileUpload.getCreatedDate()));
                        // Set the file info...
                        fileUpload.setFileID(fileID);
                        fileUpload.setFileName(fileName);
                        fileUpload.setFilePath(filePath);
                        String folderID = Configuration.root().getString("google.drive.folderid");
                        fileUpload.setFolderID(folderID);
                        // Set the URL...
                        String urlString = Configuration.root().getString("server.hostname");
                        String fullURL = urlString + "/downloadfile?ptpKey=" + adultPTP.getPtpkey() + "&fileID="
                                + fileID;
                        fileUpload.setFileURL(fullURL);
                        fileUpload.save();
                    }
                }
            }
        }
    }
} catch (Exception e) {
    e.printStackTrace();
}

我希望这能帮助下一个人。