代码之家  ›  专栏  ›  技术社区  ›  Csaba Toth

是否可以从非交互式后台调用Google的Imagen API?

  •  0
  • Csaba Toth  · 技术社区  · 1 年前

    我的目标是在QnA模式下从非交互式后端使用Imagen。 文件( https://console.cloud.google.com/vertex-ai/publishers/google/model-garden/imagetext-vqa?project=gdg-demos&cloudshell=true )使用填充承载令牌 gcloud auth print-access-token 命令如果我在云外壳中执行,我会得到一个令牌,但在非交互式后端无法使用。

        base64_string = base64_bytes.decode(ENCODING)
    
        VQA_PROMPT = "Describe the content of the image in great detail"
    
        payload = {
          "instances": [
            {
              "prompt": VQA_PROMPT,
              "image": {
                  "bytesBase64Encoded": base64_string
              }
            }
          ],
          "parameters": parameters
        }
    
        url = "https://us-central1-aiplatform.googleapis.com/v1/projects/gdg-demos/locations/us-central1/publishers/google/models/imagetext:predict"
        headers = {
            "Authorization": "Bearer {}".format(bearer_token),
            "Accept": "application/json; charset=utf-8",
        }
        json_data = requests.post(url, headers=headers, json=payload)
    

    我收到一个401 HTTP状态代码响应:

    b'{
      "error": {
        "code": 401,
        "message": "Request had invalid authentication credentials. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project.",
        "status": "UNAUTHENTICATED",
        "details": [
          {
            "@type": "type.googleapis.com/google.rpc.ErrorInfo",
            "reason": "ACCESS_TOKEN_TYPE_UNSUPPORTED",
            "metadata": {
              "service": "aiplatform.googleapis.com",
              "method": "google.cloud.aiplatform.v1.PredictionService.Predict"
            }
          }
        ]
      }
    }'
    

    我试过了 https://saturncloud.io/blog/authenticate-to-google-container-service-with-script-noninteractive-gcloud-auth-login/

    1. 向GCS认证: gcloud auth login --brief --quiet
    2. 检索刷新令牌: REFRESH_TOKEN=$(gcloud auth print-access-token)
    3. 激活刷新令牌: gcloud auth activate-refresh-token $REFRESH_TOKEN

    我打开了一个终端与JupyterLab我正在修补。我能够激活刷新令牌,并获得 Activated refresh token credentials: [***] 在第三步之后。然后我尝试使用该令牌作为Bearer令牌,但我得到了一个403 HTTP状态代码 Forbidden . 如果我执行常规(非简短和非安静),也是如此 gcloud auth打印访问令牌 在那个终端,也尝试了那个代币,但也得到了403。

    0 回复  |  直到 1 年前
        1
  •  0
  •   Csaba Toth    1 年前

    值得称赞的 Anish Nangia 谷歌指出我看错了代码。我问题中的OAuth代码不起作用。以下是我应该使用的代码: https://cloud.google.com/vertex-ai/docs/generative-ai/image/visual-question-answering#-python

    注意,当我在当地的Conda Jupyter笔记本电脑上进行实验时( https://github.com/CsabaConsulting/NextGenAI/blob/main/ImagenTest.ipynb )我仍然需要处理ADC(应用程序默认凭据),请参阅 https://cloud.google.com/docs/authentication#auth-decision-tree https://cloud.google.com/docs/authentication/application-default-credentials 然后你会得到一个 Your application is authenticating by using local Application Default Credentials. The aiplatform.googleapis.com API requires a quota project, which is not set by default. To learn how to set your quota project... ,所以有一些有趣的环,但这些环是可以解决的。

    当部署在云功能中时,您希望建立一个正确的服务帐户。示例代码: https://github.com/CsabaConsulting/NextGenAI/tree/main/imagen_test

    requirements.txt:

    functions-framework==3.*
    google-cloud-aiplatform==1.35.*
    

    main.py:

    import base64
    import functions_framework
    import vertexai
    
    from flask import jsonify
    from vertexai.vision_models import ImageQnAModel, ImageTextModel, Image
    
    PROJECT_ID = "gdg-demos"
    LOCATION = "us-central1"
    
    @functions_framework.http
    def imagen_test(request):
        """HTTP Cloud Function.
        Args:
            request (flask.Request): The request object.
            <https://flask.palletsprojects.com/en/1.1.x/api/#incoming-request-data>
        Returns:
            The response text, or any set of values that can be turned into a
            Response object using `make_response`
            <https://flask.palletsprojects.com/en/1.1.x/api/#flask.make_response>.
        """
        request_json = request.get_json(silent=True)
        request_args = request.args
    
        if request_json and 'image' in request_json:
            image_b64 = request_json['image']
        elif request_args and 'image' in request_args:
            image_b64 = request_args['image']
        else:
            image_b64 = None
    
        if not image_b64:
            return jsonify(dict(data=[]))
    
        vertexai.init(project=PROJECT_ID, location=LOCATION)
        model = ImageQnAModel.from_pretrained("imagetext@001")
    
        image_binary = base64.b64decode(image_b64)
        image = Image(image_binary)
        answers = model.ask_question(
            image=image,
            question="Describe what is on the photo in great detail, be very verbose",
            number_of_results=3,
        )
        return jsonify(dict(data=answers))