Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

In et_generator, change 'comm_size' attr from uint64 to int64 #150

Merged
merged 1 commit into from
Oct 23, 2024

Conversation

jinsun-yoo
Copy link
Contributor

@jinsun-yoo jinsun-yoo commented Sep 1, 2024

Summary

This PR makes et_generator use int64, instead of uint64 for 'comm_size' when creating collective nodes.
et_feeder parses comm_size as a int64 datatype(et_feeder.cpp:28). With the current strong datatype conversion, using a uint64 value would make attr.int64_val() parse this value as 0 (appendix 1)

Also added static_cast into uint64_t for comm_size, which #18 seems to have missed.
(@TaekyungHeo , please confirm if the lack of static_cast was intentional)

Test Plan

Created & ran the following test locally, to verify that et_generator using uint64 causes comm_size to be parsed as 0.
I did not add this test into the PR, as this is only used to verify the problem statement, not any of our code.

Generate a test ET using the following code

comm_size = 212480
output_filename = "chakra.1.et"
with open(output_filename, "wb") as et:
    encode_message(et, GlobalMetadata(version="0.0.4"))

    node = get_node("node_with_int64", COMM_COLL_NODE)
    node.attr.append(ChakraAttr(name="comm_size", int64_val=comm_size))
    encode_message(et, node)

    node = get_node("node_with_uint64", COMM_COLL_NODE)
    node.attr.append(ChakraAttr(name="comm_size", uint64_val=comm_size))
    encode_message(et, node)

To generate the following trace

{
  "version": "0.0.4"
}{
  "name": "node_with_int64",
  "type": "COMM_COLL_NODE",
  "attr": [
    {
      "name": "comm_size",
      "int64Val": "212480"
    }
  ]
}{
  "id": "1",
  "name": "node_with_uint64",
  "type": "COMM_COLL_NODE",
  "attr": [
    {
      "name": "comm_size",
      "uint64Val": "212480"
    }
  ]
}

To be tested with the following test


TEST_F(ETFeederTest, ShowUint64DoesNotWorkTest) {
  SetUp("tests/data/chakra.1.et");
  std::shared_ptr<Chakra::ETFeederNode> node;

  node = trace->lookupNode(0);
  uint64_t comm_size = node->comm_size();
  ASSERT_EQ(comm_size, 212480);

  node = trace->lookupNode(1);
  comm_size = node->comm_size();
  ASSERT_EQ(comm_size, 0);
}

Additional Notes

Appendix 1)
From et_def.pb.h:

inline ::google::protobuf::int64 AttributeProto::int64_val() const {
  // @@protoc_insertion_point(field_get:ChakraProtoMsg.AttributeProto.int64_val)
  if (has_int64_val()) {
    return value_.int64_val_;
  }
  return GOOGLE_LONGLONG(0);
}

@jinsun-yoo jinsun-yoo requested a review from a team as a code owner September 1, 2024 23:40
Copy link

github-actions bot commented Sep 1, 2024

MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅

@jinsun-yoo
Copy link
Contributor Author

Recheck

@jinsun-yoo
Copy link
Contributor Author

recheck

@jinsun-yoo jinsun-yoo closed this Sep 3, 2024
@jinsun-yoo jinsun-yoo reopened this Sep 3, 2024
@github-actions github-actions bot locked and limited conversation to collaborators Sep 3, 2024
@srinivas212
Copy link
Contributor

srinivas212 commented Oct 4, 2024

@jinsun-yoo - thanks for the PR. you need to sign the CLA before we can proceed. Can you try the following:

@mlcommons mlcommons unlocked this conversation Oct 10, 2024
@jinsun-yoo
Copy link
Contributor Author

Hi @srinivas212 , @mlcommons/wg-chakra , Please have another look at this PR. Thanks!

Copy link
Contributor

@tushar-krishna tushar-krishna left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The change looks good!

@tushar-krishna tushar-krishna merged commit 47c8154 into mlcommons:main Oct 23, 2024
9 checks passed
@github-actions github-actions bot locked and limited conversation to collaborators Oct 23, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants