代码之家 › 专栏 › 技术社区 › Csaba Toth

是否可以从非交互式后台调用Google的Imagen API?

google-generativeai large-language-model stable-diffusion google-oauth google-cloud-platform

Csaba Toth · 技术社区 · 1 年前

我的目标是在QnA模式下从非交互式后端使用Imagen。文件( https://console.cloud.google.com/vertex-ai/publishers/google/model-garden/imagetext-vqa?project=gdg-demos&cloudshell=true )使用填充承载令牌 gcloud auth print-access-token 命令如果我在云外壳中执行,我会得到一个令牌,但在非交互式后端无法使用。

    base64_string = base64_bytes.decode(ENCODING)

    VQA_PROMPT = "Describe the content of the image in great detail"

    payload = {
      "instances": [
        {
          "prompt": VQA_PROMPT,
          "image": {
              "bytesBase64Encoded": base64_string
          }
        }
      ],
      "parameters": parameters
    }

    url = "https://us-central1-aiplatform.googleapis.com/v1/projects/gdg-demos/locations/us-central1/publishers/google/models/imagetext:predict"
    headers = {
        "Authorization": "Bearer {}".format(bearer_token),
        "Accept": "application/json; charset=utf-8",
    }
    json_data = requests.post(url, headers=headers, json=payload)

我收到一个401 HTTP状态代码响应:

b'{
  "error": {
    "code": 401,
    "message": "Request had invalid authentication credentials. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project.",
    "status": "UNAUTHENTICATED",
    "details": [
      {
        "@type": "type.googleapis.com/google.rpc.ErrorInfo",
        "reason": "ACCESS_TOKEN_TYPE_UNSUPPORTED",
        "metadata": {
          "service": "aiplatform.googleapis.com",
          "method": "google.cloud.aiplatform.v1.PredictionService.Predict"
        }
      }
    ]
  }
}'

我试过了 https://saturncloud.io/blog/authenticate-to-google-container-service-with-script-noninteractive-gcloud-auth-login/

向GCS认证: gcloud auth login --brief --quiet
检索刷新令牌: REFRESH_TOKEN=$(gcloud auth print-access-token)
激活刷新令牌: gcloud auth activate-refresh-token $REFRESH_TOKEN

我打开了一个终端与JupyterLab我正在修补。我能够激活刷新令牌,并获得 Activated refresh token credentials: [***] 在第三步之后。然后我尝试使用该令牌作为Bearer令牌,但我得到了一个403 HTTP状态代码 Forbidden . 如果我执行常规(非简短和非安静),也是如此 gcloud auth打印访问令牌 在那个终端,也尝试了那个代币,但也得到了403。

0 回复 | 直到 1 年前

Csaba Toth 1 年前

值得称赞的 Anish Nangia 谷歌指出我看错了代码。我问题中的OAuth代码不起作用。以下是我应该使用的代码: https://cloud.google.com/vertex-ai/docs/generative-ai/image/visual-question-answering#-python

注意,当我在当地的Conda Jupyter笔记本电脑上进行实验时( https://github.com/CsabaConsulting/NextGenAI/blob/main/ImagenTest.ipynb )我仍然需要处理ADC(应用程序默认凭据),请参阅 https://cloud.google.com/docs/authentication#auth-decision-tree 和 https://cloud.google.com/docs/authentication/application-default-credentials 然后你会得到一个 Your application is authenticating by using local Application Default Credentials. The aiplatform.googleapis.com API requires a quota project, which is not set by default. To learn how to set your quota project... ,所以有一些有趣的环,但这些环是可以解决的。

当部署在云功能中时,您希望建立一个正确的服务帐户。示例代码: https://github.com/CsabaConsulting/NextGenAI/tree/main/imagen_test

requirements.txt:

functions-framework==3.*
google-cloud-aiplatform==1.35.*

main.py:

import base64
import functions_framework
import vertexai

from flask import jsonify
from vertexai.vision_models import ImageQnAModel, ImageTextModel, Image

PROJECT_ID = "gdg-demos"
LOCATION = "us-central1"

@functions_framework.http
def imagen_test(request):
    """HTTP Cloud Function.
    Args:
        request (flask.Request): The request object.
        <https://flask.palletsprojects.com/en/1.1.x/api/#incoming-request-data>
    Returns:
        The response text, or any set of values that can be turned into a
        Response object using `make_response`
        <https://flask.palletsprojects.com/en/1.1.x/api/#flask.make_response>.
    """
    request_json = request.get_json(silent=True)
    request_args = request.args

    if request_json and 'image' in request_json:
        image_b64 = request_json['image']
    elif request_args and 'image' in request_args:
        image_b64 = request_args['image']
    else:
        image_b64 = None

    if not image_b64:
        return jsonify(dict(data=[]))

    vertexai.init(project=PROJECT_ID, location=LOCATION)
    model = ImageQnAModel.from_pretrained("imagetext@001")

    image_binary = base64.b64decode(image_b64)
    image = Image(image_binary)
    answers = model.ask_question(
        image=image,
        question="Describe what is on the photo in great detail, be very verbose",
        number_of_results=3,
    )
    return jsonify(dict(data=answers))